2026-02-21T01:40:43.967 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T01:40:43.982 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T01:40:44.058 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-21_00:21:18-rados-wip-yuri3-testing-2026-02-19-1610-tentacle-distro-default-trial/62246 branch: wip-yuri3-testing-2026-02-19-1610-tentacle description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '62246' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-21_00:21:18-rados-wip-yuri3-testing-2026-02-19-1610-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri3-testing-2026-02-19-1610-tentacle ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 36d388f27052dd940bdbaf543bb05bdba0acc155 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 36d388f27052dd940bdbaf543bb05bdba0acc155 workunit: branch: wip-yuri3-testing-2026-02-19-1610-tentacle sha1: 36d388f27052dd940bdbaf543bb05bdba0acc155 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6877 sha1: 36d388f27052dd940bdbaf543bb05bdba0acc155 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri3-testing-2026-02-19-1610-tentacle suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_36d388f27052dd940bdbaf543bb05bdba0acc155/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 36d388f27052dd940bdbaf543bb05bdba0acc155 targets: trial009.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDDb5QOEUx1RwANUwbNB5A4KLml69KlEuaiJPH8v0GkJw0ji5z8lyBxc7Qeskh3ThlBELLOKkCJXVrRqRA5JfHo= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - osd teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_00:21:18 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T01:40:44.059 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_36d388f27052dd940bdbaf543bb05bdba0acc155/qa; will attempt to use it 2026-02-21T01:40:44.059 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_36d388f27052dd940bdbaf543bb05bdba0acc155/qa/tasks 2026-02-21T01:40:44.059 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T01:40:44.061 INFO:teuthology.task.internal:Checking packages... 2026-02-21T01:40:44.061 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '36d388f27052dd940bdbaf543bb05bdba0acc155' 2026-02-21T01:40:44.061 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T01:40:44.061 INFO:teuthology.packaging:ref: None 2026-02-21T01:40:44.061 INFO:teuthology.packaging:tag: None 2026-02-21T01:40:44.061 INFO:teuthology.packaging:branch: wip-yuri3-testing-2026-02-19-1610-tentacle 2026-02-21T01:40:44.061 INFO:teuthology.packaging:sha1: 36d388f27052dd940bdbaf543bb05bdba0acc155 2026-02-21T01:40:44.061 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri3-testing-2026-02-19-1610-tentacle 2026-02-21T01:40:44.182 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-681-g36d388f2-1jammy 2026-02-21T01:40:44.184 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T01:40:44.195 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T01:40:44.195 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T01:40:44.204 INFO:teuthology.task.internal:Saving configuration 2026-02-21T01:40:44.209 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T01:40:44.220 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T01:40:44.289 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial009.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-21_00:21:18-rados-wip-yuri3-testing-2026-02-19-1610-tentacle-distro-default-trial/62246', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 01:37:13.659874', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAILhKFl1aS9SZTVFgkYYvm/vxAPcwPOHs0hlpe0EDQMNv'} 2026-02-21T01:40:44.289 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T01:40:44.300 INFO:teuthology.task.internal:roles: ubuntu@trial009.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-21T01:40:44.300 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T01:40:44.374 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x79d5751e2ca0>, signals=[15]) 2026-02-21T01:40:44.374 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T01:40:44.385 INFO:teuthology.task.internal:Opening connections... 2026-02-21T01:40:44.385 DEBUG:teuthology.task.internal:connecting to ubuntu@trial009.front.sepia.ceph.com 2026-02-21T01:40:44.386 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial009.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T01:40:44.451 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T01:40:44.462 DEBUG:teuthology.orchestra.run.trial009:> uname -m 2026-02-21T01:40:44.529 INFO:teuthology.orchestra.run.trial009.stdout:x86_64 2026-02-21T01:40:44.529 DEBUG:teuthology.orchestra.run.trial009:> cat /etc/os-release 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:NAME="Ubuntu" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:VERSION_ID="22.04" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:VERSION_CODENAME=jammy 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:ID=ubuntu 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:ID_LIKE=debian 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-21T01:40:44.576 INFO:teuthology.orchestra.run.trial009.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-21T01:40:44.577 INFO:teuthology.orchestra.run.trial009.stdout:UBUNTU_CODENAME=jammy 2026-02-21T01:40:44.577 INFO:teuthology.lock.ops:Updating trial009.front.sepia.ceph.com on lock server 2026-02-21T01:40:44.664 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T01:40:44.676 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T01:40:44.690 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T01:40:44.690 DEBUG:teuthology.orchestra.run.trial009:> test '!' -e /home/ubuntu/cephtest 2026-02-21T01:40:44.694 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T01:40:44.703 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T01:40:44.704 DEBUG:teuthology.orchestra.run.trial009:> test -z $(ls -A /var/lib/ceph) 2026-02-21T01:40:44.740 INFO:teuthology.orchestra.run.trial009.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-21T01:40:44.741 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T01:40:44.842 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T01:40:44.861 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-21T01:40:44.861 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-21T01:40:44.861 DEBUG:teuthology.orchestra.run.trial009:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T01:40:44.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T01:40:44.865 DEBUG:teuthology.orchestra.run.trial009:> uname -r 2026-02-21T01:40:44.916 INFO:teuthology.orchestra.run.trial009.stdout:5.15.0-170-generic 2026-02-21T01:40:44.916 INFO:teuthology.task.kernel:Running kernel on trial009: 5.15.0-170-generic 2026-02-21T01:40:44.916 DEBUG:teuthology.orchestra.run.trial009:> sudo apt-get clean 2026-02-21T01:40:44.978 DEBUG:teuthology.orchestra.run.trial009:> sudo apt-get update 2026-02-21T01:40:45.412 INFO:teuthology.orchestra.run.trial009.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T01:40:45.484 INFO:teuthology.orchestra.run.trial009.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-21T01:40:45.761 INFO:teuthology.orchestra.run.trial009.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-21T01:40:45.882 INFO:teuthology.orchestra.run.trial009.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-21T01:40:45.995 INFO:teuthology.orchestra.run.trial009.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-21T01:40:46.323 INFO:teuthology.orchestra.run.trial009.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-21T01:40:46.332 INFO:teuthology.orchestra.run.trial009.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-21T01:40:46.333 INFO:teuthology.orchestra.run.trial009.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-21T01:40:46.361 INFO:teuthology.orchestra.run.trial009.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-21T01:40:46.367 INFO:teuthology.orchestra.run.trial009.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-21T01:40:46.368 INFO:teuthology.orchestra.run.trial009.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-21T01:40:46.496 INFO:teuthology.orchestra.run.trial009.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-21T01:40:46.514 INFO:teuthology.orchestra.run.trial009.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-21T01:40:46.586 INFO:teuthology.orchestra.run.trial009.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-21T01:40:46.591 INFO:teuthology.orchestra.run.trial009.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-21T01:40:46.591 INFO:teuthology.orchestra.run.trial009.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-21T01:40:46.619 INFO:teuthology.orchestra.run.trial009.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-21T01:40:46.620 INFO:teuthology.orchestra.run.trial009.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-21T01:40:46.742 INFO:teuthology.orchestra.run.trial009.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-21T01:40:50.261 INFO:teuthology.orchestra.run.trial009.stdout:Fetched 22.6 MB in 3s (8659 kB/s) 2026-02-21T01:40:50.783 INFO:teuthology.orchestra.run.trial009.stdout:Reading package lists... 2026-02-21T01:40:50.796 DEBUG:teuthology.orchestra.run.trial009:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-21T01:40:50.829 INFO:teuthology.orchestra.run.trial009.stdout:Reading package lists... 2026-02-21T01:40:50.934 INFO:teuthology.orchestra.run.trial009.stdout:Building dependency tree... 2026-02-21T01:40:50.935 INFO:teuthology.orchestra.run.trial009.stdout:Reading state information... 2026-02-21T01:40:51.016 INFO:teuthology.orchestra.run.trial009.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-21T01:40:51.017 INFO:teuthology.orchestra.run.trial009.stdout:linux-image-generic set to manually installed. 2026-02-21T01:40:51.017 INFO:teuthology.orchestra.run.trial009.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T01:40:51.017 INFO:teuthology.orchestra.run.trial009.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T01:40:51.017 INFO:teuthology.orchestra.run.trial009.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T01:40:51.017 INFO:teuthology.orchestra.run.trial009.stdout: sg3-utils-udev 2026-02-21T01:40:51.017 INFO:teuthology.orchestra.run.trial009.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T01:40:51.036 INFO:teuthology.orchestra.run.trial009.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T01:40:51.037 DEBUG:teuthology.orchestra.run.trial009:> dpkg -s linux-image-generic 2026-02-21T01:40:51.090 INFO:teuthology.orchestra.run.trial009.stdout:Package: linux-image-generic 2026-02-21T01:40:51.090 INFO:teuthology.orchestra.run.trial009.stdout:Status: install ok installed 2026-02-21T01:40:51.090 INFO:teuthology.orchestra.run.trial009.stdout:Priority: optional 2026-02-21T01:40:51.090 INFO:teuthology.orchestra.run.trial009.stdout:Section: kernel 2026-02-21T01:40:51.090 INFO:teuthology.orchestra.run.trial009.stdout:Installed-Size: 21 2026-02-21T01:40:51.090 INFO:teuthology.orchestra.run.trial009.stdout:Maintainer: Ubuntu Kernel Team 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Architecture: amd64 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Source: linux-meta 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Version: 5.15.0.170.159 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Recommends: thermald 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout:Description: Generic Linux kernel image 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout: This package will always depend on the latest generic kernel image 2026-02-21T01:40:51.091 INFO:teuthology.orchestra.run.trial009.stdout: available. 2026-02-21T01:40:51.092 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T01:40:51.092 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T01:40:51.092 DEBUG:teuthology.orchestra.run.trial009:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T01:40:51.140 INFO:teuthology.orchestra.run.trial009.stdout:ttyS1 2026-02-21T01:40:51.182 DEBUG:teuthology.parallel:result is None 2026-02-21T01:40:51.182 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T01:40:51.189 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T01:40:51.189 DEBUG:teuthology.orchestra.run.trial009:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T01:40:51.227 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T01:40:51.241 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T01:40:51.257 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T01:40:51.258 DEBUG:teuthology.orchestra.run.trial009:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T01:40:51.274 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T01:40:51.285 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T01:40:51.285 DEBUG:teuthology.orchestra.run.trial009:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T01:40:51.324 INFO:teuthology.orchestra.run.trial009.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:40:51.331 INFO:teuthology.orchestra.run.trial009.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:40:51.332 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T01:40:51.344 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T01:40:51.344 DEBUG:teuthology.orchestra.run.trial009:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T01:40:51.387 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T01:40:51.414 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T01:40:51.414 DEBUG:teuthology.orchestra.run.trial009:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T01:40:51.441 DEBUG:teuthology.orchestra.run.trial009:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T01:40:51.490 DEBUG:teuthology.orchestra.run.trial009:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T01:40:51.537 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:40:51.537 DEBUG:teuthology.orchestra.run.trial009:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T01:40:51.587 DEBUG:teuthology.orchestra.run.trial009:> sudo service rsyslog restart 2026-02-21T01:40:51.645 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T01:40:51.657 INFO:teuthology.task.internal:Starting timer... 2026-02-21T01:40:51.657 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T01:40:51.673 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T01:40:51.685 DEBUG:teuthology.task.selinux:Excluding trial009: OS 'ubuntu' does not support SELinux 2026-02-21T01:40:51.685 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T01:40:51.685 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-21T01:40:51.685 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T01:40:51.685 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T01:40:51.696 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T01:40:51.703 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T01:40:51.703 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial009.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T01:43:10.158 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial009.front.sepia.ceph.com')] 2026-02-21T01:43:10.159 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial009.front.sepia.ceph.com' 2026-02-21T01:43:10.161 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial009.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T01:43:10.268 DEBUG:teuthology.orchestra.run.trial009:> true 2026-02-21T01:43:10.517 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial009.front.sepia.ceph.com' 2026-02-21T01:43:10.517 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T01:43:10.536 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T01:43:10.536 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T01:43:10.536 DEBUG:teuthology.orchestra.run.trial009:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T01:43:10.582 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Command line: ntpd -gq 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: ---------------------------------------------------- 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: ntp-4 is maintained by Network Time Foundation, 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: corporation. Support and training for ntp-4 are 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: available at https://www.nwtime.org/support 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: ---------------------------------------------------- 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: proto: precision = 0.020 usec (-25) 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: basedate set to 2022-02-04 2026-02-21T01:43:10.583 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: gps base set to 2022-02-06 (week 2196) 2026-02-21T01:43:10.584 INFO:teuthology.orchestra.run.trial009.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T01:43:10.584 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-21T01:43:10.584 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: restrict ::: KOD does nothing without LIMITED. 2026-02-21T01:43:10.584 INFO:teuthology.orchestra.run.trial009.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-21T01:43:10.584 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listen and drop on 0 v6wildcard [::]:123 2026-02-21T01:43:10.585 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-21T01:43:10.585 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listen normally on 2 lo 127.0.0.1:123 2026-02-21T01:43:10.585 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listen normally on 3 enp1s0f0 10.20.193.9:123 2026-02-21T01:43:10.585 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listen normally on 4 lo [::1]:123 2026-02-21T01:43:10.585 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:5fcc%2]:123 2026-02-21T01:43:10.585 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:10 ntpd[9404]: Listening on routing socket on fd #22 for interface updates 2026-02-21T01:43:12.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:12 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:12.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:12 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:12.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:12 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:12.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:12 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:12.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:12 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:12.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:12 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:13.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:13.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:13.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:13.585 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:13.585 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:13.585 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:13.593 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:13.593 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:13.593 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:13.594 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:13.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:13.618 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:13.618 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:13.618 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:13.659 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:13.659 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:13.659 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:13.659 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:13 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:14.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:14 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:14.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:14 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:14.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:14 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:14.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:14 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:14.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:14 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:14.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:14 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:15.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:15.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:15.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:15.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:15.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:15.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:15.592 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:15.592 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:15.592 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:15.592 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:15.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:15.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:15.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:15.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:15.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:15.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:15.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:15.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:15 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:16.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:16 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:16.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:16 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:16.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:16 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:16.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:16 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:16.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:16 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:16.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:16 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:17.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:17.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:17.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:17.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:17.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:17.584 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:17.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:17.617 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:17.618 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:17.618 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:17.659 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:17.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:17.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:17.660 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:17 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:18.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:18.583 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:18.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-21T01:43:18.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: can't open /var/log/ntpstats/rawstats.20260221: Permission denied 2026-02-21T01:43:18.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-21T01:43:18.586 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: can't open /var/log/ntpstats/peerstats.20260221: Permission denied 2026-02-21T01:43:18.587 INFO:teuthology.orchestra.run.trial009.stdout:21 Feb 01:43:18 ntpd[9404]: ntpd: time set +0.036631 s 2026-02-21T01:43:18.587 INFO:teuthology.orchestra.run.trial009.stdout:ntpd: time set +0.036631s 2026-02-21T01:43:18.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-21T01:43:18.587 INFO:teuthology.orchestra.run.trial009.stderr:21 Feb 01:43:18 ntpd[9404]: can't open /var/log/ntpstats/loopstats.20260221: Permission denied 2026-02-21T01:43:18.639 INFO:teuthology.orchestra.run.trial009.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T01:43:18.639 INFO:teuthology.orchestra.run.trial009.stdout:============================================================================== 2026-02-21T01:43:18.640 INFO:teuthology.orchestra.run.trial009.stdout: 23.157.160.168 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T01:43:18.640 INFO:teuthology.orchestra.run.trial009.stdout: LAX.CALTICK.NET .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T01:43:18.640 INFO:teuthology.orchestra.run.trial009.stdout: 216.144.228.179 .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T01:43:18.640 INFO:teuthology.orchestra.run.trial009.stdout: 144.202.41.38 ( .INIT. 16 u - 64 0 0.000 +0.000 0.000 2026-02-21T01:43:18.640 INFO:teuthology.run_tasks:Running task install... 2026-02-21T01:43:18.655 DEBUG:teuthology.task.install:project ceph 2026-02-21T01:43:18.655 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '36d388f27052dd940bdbaf543bb05bdba0acc155'}} 2026-02-21T01:43:18.655 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '36d388f27052dd940bdbaf543bb05bdba0acc155'} 2026-02-21T01:43:18.655 INFO:teuthology.task.install:Using flavor: default 2026-02-21T01:43:18.665 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T01:43:18.666 INFO:teuthology.task.install:extra packages: [] 2026-02-21T01:43:18.666 DEBUG:teuthology.orchestra.run.trial009:> sudo apt-key list | grep Ceph 2026-02-21T01:43:18.731 INFO:teuthology.orchestra.run.trial009.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-21T01:43:18.760 INFO:teuthology.orchestra.run.trial009.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-21T01:43:18.760 INFO:teuthology.orchestra.run.trial009.stdout:uid [ unknown] Ceph.com (release key) 2026-02-21T01:43:18.761 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-21T01:43:18.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=36d388f27052dd940bdbaf543bb05bdba0acc155 2026-02-21T01:43:18.950 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default/ 2026-02-21T01:43:18.950 INFO:teuthology.task.install.deb:Package version is 20.2.0-681-g36d388f2-1jammy 2026-02-21T01:43:19.114 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:43:19.114 DEBUG:teuthology.orchestra.run.trial009:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-21T01:43:19.124 DEBUG:teuthology.orchestra.run.trial009:> sudo apt-get update 2026-02-21T01:43:19.296 INFO:teuthology.orchestra.run.trial009.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy InRelease 2026-02-21T01:43:19.314 INFO:teuthology.orchestra.run.trial009.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy Release [7680 B] 2026-02-21T01:43:19.332 INFO:teuthology.orchestra.run.trial009.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-21T01:43:19.350 INFO:teuthology.orchestra.run.trial009.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.8 kB] 2026-02-21T01:43:19.556 INFO:teuthology.orchestra.run.trial009.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-21T01:43:19.626 INFO:teuthology.orchestra.run.trial009.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-21T01:43:19.697 INFO:teuthology.orchestra.run.trial009.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-21T01:43:19.769 INFO:teuthology.orchestra.run.trial009.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-21T01:43:19.922 INFO:teuthology.orchestra.run.trial009.stdout:Fetched 26.5 kB in 1s (42.4 kB/s) 2026-02-21T01:43:20.443 INFO:teuthology.orchestra.run.trial009.stdout:Reading package lists... 2026-02-21T01:43:20.456 DEBUG:teuthology.orchestra.run.trial009:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=20.2.0-681-g36d388f2-1jammy cephadm=20.2.0-681-g36d388f2-1jammy ceph-mds=20.2.0-681-g36d388f2-1jammy ceph-mgr=20.2.0-681-g36d388f2-1jammy ceph-common=20.2.0-681-g36d388f2-1jammy ceph-fuse=20.2.0-681-g36d388f2-1jammy ceph-test=20.2.0-681-g36d388f2-1jammy ceph-volume=20.2.0-681-g36d388f2-1jammy radosgw=20.2.0-681-g36d388f2-1jammy python3-rados=20.2.0-681-g36d388f2-1jammy python3-rgw=20.2.0-681-g36d388f2-1jammy python3-cephfs=20.2.0-681-g36d388f2-1jammy python3-rbd=20.2.0-681-g36d388f2-1jammy libcephfs2=20.2.0-681-g36d388f2-1jammy libcephfs-dev=20.2.0-681-g36d388f2-1jammy librados2=20.2.0-681-g36d388f2-1jammy librbd1=20.2.0-681-g36d388f2-1jammy rbd-fuse=20.2.0-681-g36d388f2-1jammy 2026-02-21T01:43:20.488 INFO:teuthology.orchestra.run.trial009.stdout:Reading package lists... 2026-02-21T01:43:20.594 INFO:teuthology.orchestra.run.trial009.stdout:Building dependency tree... 2026-02-21T01:43:20.595 INFO:teuthology.orchestra.run.trial009.stdout:Reading state information... 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout:The following packages were automatically installed and are no longer required: 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout: sg3-utils-udev 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout:The following additional packages will be installed: 2026-02-21T01:43:20.685 INFO:teuthology.orchestra.run.trial009.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: libcephfs-daemon libcephfs-proxy2 libdouble-conversion3 libfuse2 libjq1 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: liblttng-ust1 libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: libqt5dbus5 libqt5network5 libradosstriper1 librdkafka1 librgw2 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli python-asyncssh-doc 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-asyncssh python3-cachetools python3-ceph-argparse 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-ceph-common python3-cheroot python3-cherrypy3 python3-google-auth 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-jaraco.functools python3-jaraco.text python3-joblib 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-kubernetes python3-natsort python3-pluggy python3-portend 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-pytest python3-repoze.lru python3-requests-oauthlib python3-routes 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-rsa python3-simplejson python3-sklearn python3-sklearn-lib 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-tempora python3-threadpoolctl python3-toml python3-wcwidth 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-webob python3-websocket python3-zc.lockfile qttranslations5-l10n 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: socat xmlstarlet 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout:Suggested packages: 2026-02-21T01:43:20.686 INFO:teuthology.orchestra.run.trial009.stdout: python3-influxdb liblua5.3-dev luarocks python-natsort-doc python-psutil-doc 2026-02-21T01:43:20.687 INFO:teuthology.orchestra.run.trial009.stdout: subversion python-pygments-doc ttf-bitstream-vera python3-paste python3-dap 2026-02-21T01:43:20.687 INFO:teuthology.orchestra.run.trial009.stdout: python-sklearn-doc ipython3 python-webob-doc 2026-02-21T01:43:20.687 INFO:teuthology.orchestra.run.trial009.stdout:Recommended packages: 2026-02-21T01:43:20.687 INFO:teuthology.orchestra.run.trial009.stdout: btrfs-tools 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout:The following NEW packages will be installed: 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: libcephfs-daemon libcephfs-dev libcephfs-proxy2 libcephfs2 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 libnbd0 liboath0 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python3-asyncssh 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-natsort 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-pygments python3-pytest python3-rados python3-rbd python3-repoze.lru 2026-02-21T01:43:20.718 INFO:teuthology.orchestra.run.trial009.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2026-02-21T01:43:20.719 INFO:teuthology.orchestra.run.trial009.stdout: python3-simplejson python3-sklearn python3-sklearn-lib python3-tempora 2026-02-21T01:43:20.719 INFO:teuthology.orchestra.run.trial009.stdout: python3-threadpoolctl python3-toml python3-wcwidth python3-webob 2026-02-21T01:43:20.719 INFO:teuthology.orchestra.run.trial009.stdout: python3-websocket python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2026-02-21T01:43:20.719 INFO:teuthology.orchestra.run.trial009.stdout: socat xmlstarlet 2026-02-21T01:43:20.837 INFO:teuthology.orchestra.run.trial009.stdout:0 upgraded, 86 newly installed, 0 to remove and 13 not upgraded. 2026-02-21T01:43:20.837 INFO:teuthology.orchestra.run.trial009.stdout:Need to get 280 MB of archives. 2026-02-21T01:43:20.837 INFO:teuthology.orchestra.run.trial009.stdout:After this operation, 1119 MB of additional disk space will be used. 2026-02-21T01:43:20.837 INFO:teuthology.orchestra.run.trial009.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 20.2.0-681-g36d388f2-1jammy [3569 kB] 2026-02-21T01:43:21.050 INFO:teuthology.orchestra.run.trial009.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 20.2.0-681-g36d388f2-1jammy [2867 kB] 2026-02-21T01:43:21.073 INFO:teuthology.orchestra.run.trial009.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-21T01:43:21.137 INFO:teuthology.orchestra.run.trial009.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 20.2.0-681-g36d388f2-1jammy [833 kB] 2026-02-21T01:43:21.168 INFO:teuthology.orchestra.run.trial009.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 20.2.0-681-g36d388f2-1jammy [364 kB] 2026-02-21T01:43:21.175 INFO:teuthology.orchestra.run.trial009.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 20.2.0-681-g36d388f2-1jammy [32.9 kB] 2026-02-21T01:43:21.176 INFO:teuthology.orchestra.run.trial009.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 20.2.0-681-g36d388f2-1jammy [184 kB] 2026-02-21T01:43:21.179 INFO:teuthology.orchestra.run.trial009.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 20.2.0-681-g36d388f2-1jammy [83.9 kB] 2026-02-21T01:43:21.180 INFO:teuthology.orchestra.run.trial009.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 20.2.0-681-g36d388f2-1jammy [341 kB] 2026-02-21T01:43:21.187 INFO:teuthology.orchestra.run.trial009.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 20.2.0-681-g36d388f2-1jammy [8686 kB] 2026-02-21T01:43:21.413 INFO:teuthology.orchestra.run.trial009.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-21T01:43:21.424 INFO:teuthology.orchestra.run.trial009.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-21T01:43:21.447 INFO:teuthology.orchestra.run.trial009.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 20.2.0-681-g36d388f2-1jammy [112 kB] 2026-02-21T01:43:21.447 INFO:teuthology.orchestra.run.trial009.stdout:Get:14 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 20.2.0-681-g36d388f2-1jammy [261 kB] 2026-02-21T01:43:21.452 INFO:teuthology.orchestra.run.trial009.stdout:Get:15 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 20.2.0-681-g36d388f2-1jammy [29.3 MB] 2026-02-21T01:43:21.494 INFO:teuthology.orchestra.run.trial009.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-21T01:43:21.696 INFO:teuthology.orchestra.run.trial009.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-21T01:43:21.706 INFO:teuthology.orchestra.run.trial009.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-21T01:43:21.733 INFO:teuthology.orchestra.run.trial009.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-21T01:43:21.741 INFO:teuthology.orchestra.run.trial009.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-21T01:43:21.743 INFO:teuthology.orchestra.run.trial009.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-21T01:43:21.744 INFO:teuthology.orchestra.run.trial009.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-21T01:43:21.745 INFO:teuthology.orchestra.run.trial009.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-21T01:43:21.761 INFO:teuthology.orchestra.run.trial009.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-21T01:43:21.762 INFO:teuthology.orchestra.run.trial009.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-21T01:43:21.767 INFO:teuthology.orchestra.run.trial009.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-21T01:43:21.835 INFO:teuthology.orchestra.run.trial009.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-21T01:43:21.835 INFO:teuthology.orchestra.run.trial009.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-21T01:43:21.836 INFO:teuthology.orchestra.run.trial009.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-21T01:43:21.836 INFO:teuthology.orchestra.run.trial009.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-21T01:43:21.836 INFO:teuthology.orchestra.run.trial009.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-21T01:43:21.836 INFO:teuthology.orchestra.run.trial009.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-21T01:43:21.837 INFO:teuthology.orchestra.run.trial009.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-21T01:43:21.840 INFO:teuthology.orchestra.run.trial009.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-21T01:43:21.840 INFO:teuthology.orchestra.run.trial009.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-21T01:43:21.911 INFO:teuthology.orchestra.run.trial009.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-21T01:43:21.913 INFO:teuthology.orchestra.run.trial009.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-21T01:43:21.913 INFO:teuthology.orchestra.run.trial009.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-21T01:43:21.914 INFO:teuthology.orchestra.run.trial009.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-21T01:43:21.939 INFO:teuthology.orchestra.run.trial009.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-21T01:43:21.941 INFO:teuthology.orchestra.run.trial009.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-21T01:43:21.941 INFO:teuthology.orchestra.run.trial009.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-21T01:43:21.999 INFO:teuthology.orchestra.run.trial009.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-21T01:43:22.000 INFO:teuthology.orchestra.run.trial009.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-21T01:43:22.000 INFO:teuthology.orchestra.run.trial009.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-21T01:43:22.058 INFO:teuthology.orchestra.run.trial009.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-21T01:43:22.058 INFO:teuthology.orchestra.run.trial009.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-21T01:43:22.058 INFO:teuthology.orchestra.run.trial009.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-21T01:43:22.061 INFO:teuthology.orchestra.run.trial009.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-21T01:43:22.062 INFO:teuthology.orchestra.run.trial009.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-21T01:43:22.064 INFO:teuthology.orchestra.run.trial009.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-21T01:43:22.130 INFO:teuthology.orchestra.run.trial009.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-21T01:43:22.132 INFO:teuthology.orchestra.run.trial009.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-21T01:43:22.135 INFO:teuthology.orchestra.run.trial009.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-21T01:43:22.138 INFO:teuthology.orchestra.run.trial009.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-21T01:43:22.202 INFO:teuthology.orchestra.run.trial009.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-21T01:43:22.202 INFO:teuthology.orchestra.run.trial009.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-21T01:43:22.203 INFO:teuthology.orchestra.run.trial009.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-21T01:43:22.204 INFO:teuthology.orchestra.run.trial009.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-21T01:43:22.204 INFO:teuthology.orchestra.run.trial009.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-21T01:43:22.211 INFO:teuthology.orchestra.run.trial009.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-21T01:43:22.218 INFO:teuthology.orchestra.run.trial009.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 20.2.0-681-g36d388f2-1jammy [5417 kB] 2026-02-21T01:43:22.274 INFO:teuthology.orchestra.run.trial009.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-21T01:43:22.276 INFO:teuthology.orchestra.run.trial009.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-21T01:43:22.277 INFO:teuthology.orchestra.run.trial009.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-21T01:43:22.277 INFO:teuthology.orchestra.run.trial009.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-21T01:43:22.321 INFO:teuthology.orchestra.run.trial009.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 20.2.0-681-g36d388f2-1jammy [247 kB] 2026-02-21T01:43:22.325 INFO:teuthology.orchestra.run.trial009.stdout:Get:68 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 20.2.0-681-g36d388f2-1jammy [124 kB] 2026-02-21T01:43:22.327 INFO:teuthology.orchestra.run.trial009.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 20.2.0-681-g36d388f2-1jammy [906 kB] 2026-02-21T01:43:22.360 INFO:teuthology.orchestra.run.trial009.stdout:Get:70 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 20.2.0-681-g36d388f2-1jammy [6405 kB] 2026-02-21T01:43:22.521 INFO:teuthology.orchestra.run.trial009.stdout:Get:71 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 20.2.0-681-g36d388f2-1jammy [21.7 MB] 2026-02-21T01:43:23.128 INFO:teuthology.orchestra.run.trial009.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 20.2.0-681-g36d388f2-1jammy [14.1 kB] 2026-02-21T01:43:23.128 INFO:teuthology.orchestra.run.trial009.stdout:Get:73 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 20.2.0-681-g36d388f2-1jammy [957 kB] 2026-02-21T01:43:23.141 INFO:teuthology.orchestra.run.trial009.stdout:Get:74 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 20.2.0-681-g36d388f2-1jammy [2345 kB] 2026-02-21T01:43:23.230 INFO:teuthology.orchestra.run.trial009.stdout:Get:75 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 20.2.0-681-g36d388f2-1jammy [1049 kB] 2026-02-21T01:43:23.248 INFO:teuthology.orchestra.run.trial009.stdout:Get:76 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 20.2.0-681-g36d388f2-1jammy [179 kB] 2026-02-21T01:43:23.251 INFO:teuthology.orchestra.run.trial009.stdout:Get:77 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 20.2.0-681-g36d388f2-1jammy [45.5 MB] 2026-02-21T01:43:24.457 INFO:teuthology.orchestra.run.trial009.stdout:Get:78 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 20.2.0-681-g36d388f2-1jammy [8625 kB] 2026-02-21T01:43:24.690 INFO:teuthology.orchestra.run.trial009.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 20.2.0-681-g36d388f2-1jammy [14.2 kB] 2026-02-21T01:43:24.690 INFO:teuthology.orchestra.run.trial009.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 20.2.0-681-g36d388f2-1jammy [99.5 MB] 2026-02-21T01:43:30.015 INFO:teuthology.orchestra.run.trial009.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 20.2.0-681-g36d388f2-1jammy [135 kB] 2026-02-21T01:43:30.016 INFO:teuthology.orchestra.run.trial009.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-daemon amd64 20.2.0-681-g36d388f2-1jammy [43.3 kB] 2026-02-21T01:43:30.016 INFO:teuthology.orchestra.run.trial009.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-proxy2 amd64 20.2.0-681-g36d388f2-1jammy [30.7 kB] 2026-02-21T01:43:30.016 INFO:teuthology.orchestra.run.trial009.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 20.2.0-681-g36d388f2-1jammy [41.5 kB] 2026-02-21T01:43:30.017 INFO:teuthology.orchestra.run.trial009.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 20.2.0-681-g36d388f2-1jammy [25.1 MB] 2026-02-21T01:43:31.928 INFO:teuthology.orchestra.run.trial009.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri3-testing-2026-02-19-1610-tentacle/36d388f27052dd940bdbaf543bb05bdba0acc155/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 20.2.0-681-g36d388f2-1jammy [97.5 kB] 2026-02-21T01:43:32.217 INFO:teuthology.orchestra.run.trial009.stdout:Fetched 280 MB in 11s (25.0 MB/s) 2026-02-21T01:43:32.233 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-21T01:43:32.390 INFO:teuthology.orchestra.run.trial009.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-21T01:43:32.391 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../00-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-21T01:43:32.393 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T01:43:32.414 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-21T01:43:32.427 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../01-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-21T01:43:32.427 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T01:43:32.443 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-21T01:43:32.455 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../02-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-21T01:43:32.455 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T01:43:32.475 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-21T01:43:32.483 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../03-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T01:43:32.488 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T01:43:32.516 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-21T01:43:32.524 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../04-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T01:43:32.524 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T01:43:32.541 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-21T01:43:32.549 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../05-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-21T01:43:32.550 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T01:43:32.569 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-21T01:43:32.577 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../06-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-21T01:43:32.578 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T01:43:32.594 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package librados2. 2026-02-21T01:43:32.602 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../07-librados2_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.602 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking librados2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.642 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libnbd0. 2026-02-21T01:43:32.650 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../08-libnbd0_1.10.5-1_amd64.deb ... 2026-02-21T01:43:32.651 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-21T01:43:32.665 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package librbd1. 2026-02-21T01:43:32.673 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../09-librbd1_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.674 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking librbd1 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.710 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libcephfs2. 2026-02-21T01:43:32.719 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../10-libcephfs2_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.719 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libcephfs2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.738 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-rados. 2026-02-21T01:43:32.747 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../11-python3-rados_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.747 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-rados (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.763 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-21T01:43:32.772 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../12-python3-ceph-argparse_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:32.772 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-ceph-argparse (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.786 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-cephfs. 2026-02-21T01:43:32.794 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../13-python3-cephfs_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.794 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-cephfs (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.809 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-21T01:43:32.817 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../14-python3-ceph-common_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:32.818 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-ceph-common (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.836 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-21T01:43:32.844 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../15-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-21T01:43:32.845 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T01:43:32.860 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-prettytable. 2026-02-21T01:43:32.868 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../16-python3-prettytable_2.5.0-2_all.deb ... 2026-02-21T01:43:32.869 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-21T01:43:32.883 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-rbd. 2026-02-21T01:43:32.891 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../17-python3-rbd_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.892 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-rbd (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:32.909 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-21T01:43:32.917 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../18-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-21T01:43:32.918 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T01:43:32.936 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package librgw2. 2026-02-21T01:43:32.944 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../19-librgw2_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:32.945 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking librgw2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.037 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-rgw. 2026-02-21T01:43:33.046 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../20-python3-rgw_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.046 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-rgw (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.061 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-21T01:43:33.069 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../21-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-21T01:43:33.070 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T01:43:33.084 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libradosstriper1. 2026-02-21T01:43:33.092 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../22-libradosstriper1_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.093 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libradosstriper1 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.110 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-common. 2026-02-21T01:43:33.118 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../23-ceph-common_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.118 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-common (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.360 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-base. 2026-02-21T01:43:33.369 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../24-ceph-base_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.374 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-base (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.438 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-21T01:43:33.447 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../25-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-21T01:43:33.447 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-21T01:43:33.461 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-cheroot. 2026-02-21T01:43:33.469 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../26-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-21T01:43:33.470 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T01:43:33.486 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-21T01:43:33.495 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../27-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-21T01:43:33.495 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-21T01:43:33.509 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-21T01:43:33.518 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../28-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-21T01:43:33.518 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-21T01:43:33.532 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-21T01:43:33.541 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../29-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-21T01:43:33.541 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-21T01:43:33.555 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-tempora. 2026-02-21T01:43:33.563 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../30-python3-tempora_4.1.2-1_all.deb ... 2026-02-21T01:43:33.564 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-21T01:43:33.579 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-portend. 2026-02-21T01:43:33.587 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../31-python3-portend_3.0.0-1_all.deb ... 2026-02-21T01:43:33.588 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-21T01:43:33.601 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-21T01:43:33.610 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../32-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-21T01:43:33.610 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-21T01:43:33.625 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-21T01:43:33.633 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../33-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-21T01:43:33.633 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-21T01:43:33.657 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-natsort. 2026-02-21T01:43:33.665 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../34-python3-natsort_8.0.2-1_all.deb ... 2026-02-21T01:43:33.665 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-21T01:43:33.681 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-21T01:43:33.689 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../35-ceph-mgr-modules-core_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:33.690 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mgr-modules-core (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.716 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-21T01:43:33.724 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../36-libsqlite3-mod-ceph_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.725 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libsqlite3-mod-ceph (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.739 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mgr. 2026-02-21T01:43:33.748 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../37-ceph-mgr_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.748 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mgr (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.769 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mon. 2026-02-21T01:43:33.777 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../38-ceph-mon_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.778 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mon (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:33.836 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-21T01:43:33.844 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../39-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-21T01:43:33.845 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T01:43:33.861 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-osd. 2026-02-21T01:43:33.869 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../40-ceph-osd_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:33.869 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-osd (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.017 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph. 2026-02-21T01:43:34.026 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../41-ceph_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:34.026 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.040 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-fuse. 2026-02-21T01:43:34.048 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../42-ceph-fuse_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:34.048 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-fuse (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.070 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mds. 2026-02-21T01:43:34.078 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../43-ceph-mds_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:34.079 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mds (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.109 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package cephadm. 2026-02-21T01:43:34.117 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../44-cephadm_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:34.118 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking cephadm (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.135 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-21T01:43:34.143 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../45-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T01:43:34.143 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T01:43:34.163 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-21T01:43:34.172 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../46-ceph-mgr-cephadm_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:34.172 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mgr-cephadm (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.192 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-21T01:43:34.200 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../47-python3-repoze.lru_0.7-2_all.deb ... 2026-02-21T01:43:34.201 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-21T01:43:34.216 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-routes. 2026-02-21T01:43:34.224 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../48-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-21T01:43:34.225 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T01:43:34.245 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-21T01:43:34.253 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../49-ceph-mgr-dashboard_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:34.253 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mgr-dashboard (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:34.689 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-21T01:43:34.699 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../50-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-21T01:43:34.699 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T01:43:34.737 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-joblib. 2026-02-21T01:43:34.745 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../51-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-21T01:43:34.746 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T01:43:34.773 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-21T01:43:34.782 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../52-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-21T01:43:34.782 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-21T01:43:34.797 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-sklearn. 2026-02-21T01:43:34.805 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../53-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-21T01:43:34.806 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T01:43:34.888 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-21T01:43:34.897 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../54-ceph-mgr-diskprediction-local_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:34.898 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mgr-diskprediction-local (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:35.065 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-cachetools. 2026-02-21T01:43:35.073 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../55-python3-cachetools_5.0.0-1_all.deb ... 2026-02-21T01:43:35.074 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-21T01:43:35.089 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-rsa. 2026-02-21T01:43:35.097 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../56-python3-rsa_4.8-1_all.deb ... 2026-02-21T01:43:35.097 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-21T01:43:35.116 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-google-auth. 2026-02-21T01:43:35.124 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../57-python3-google-auth_1.5.1-3_all.deb ... 2026-02-21T01:43:35.125 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-21T01:43:35.143 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-21T01:43:35.151 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../58-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-21T01:43:35.152 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T01:43:35.168 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-websocket. 2026-02-21T01:43:35.176 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../59-python3-websocket_1.2.3-1_all.deb ... 2026-02-21T01:43:35.177 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-21T01:43:35.194 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-21T01:43:35.202 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../60-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-21T01:43:35.203 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T01:43:35.298 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-21T01:43:35.307 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../61-ceph-mgr-k8sevents_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:35.307 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-mgr-k8sevents (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:35.322 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-21T01:43:35.330 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../62-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-21T01:43:35.330 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T01:43:35.346 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-21T01:43:35.355 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../63-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T01:43:35.355 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T01:43:35.371 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package jq. 2026-02-21T01:43:35.379 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../64-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-21T01:43:35.379 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-21T01:43:35.393 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package socat. 2026-02-21T01:43:35.401 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../65-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-21T01:43:35.402 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-21T01:43:35.423 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package xmlstarlet. 2026-02-21T01:43:35.432 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../66-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-21T01:43:35.433 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-21T01:43:35.470 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-test. 2026-02-21T01:43:35.478 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../67-ceph-test_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:35.479 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-test (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:36.345 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package ceph-volume. 2026-02-21T01:43:36.353 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../68-ceph-volume_20.2.0-681-g36d388f2-1jammy_all.deb ... 2026-02-21T01:43:36.354 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking ceph-volume (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:36.377 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libcephfs-daemon. 2026-02-21T01:43:36.385 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../69-libcephfs-daemon_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:36.386 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libcephfs-daemon (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:36.400 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libcephfs-proxy2. 2026-02-21T01:43:36.409 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../70-libcephfs-proxy2_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:36.409 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libcephfs-proxy2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:36.423 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-21T01:43:36.431 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../71-libcephfs-dev_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:36.432 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking libcephfs-dev (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:36.448 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package nvme-cli. 2026-02-21T01:43:36.456 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../72-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-21T01:43:36.457 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T01:43:36.489 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-21T01:43:36.497 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../73-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-21T01:43:36.497 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T01:43:36.527 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-21T01:43:36.535 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../74-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-21T01:43:36.536 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-21T01:43:36.550 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-pluggy. 2026-02-21T01:43:36.558 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../75-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-21T01:43:36.559 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-21T01:43:36.574 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-psutil. 2026-02-21T01:43:36.582 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../76-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-21T01:43:36.583 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-21T01:43:36.601 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-py. 2026-02-21T01:43:36.609 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../77-python3-py_1.10.0-1_all.deb ... 2026-02-21T01:43:36.610 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-21T01:43:36.629 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-pygments. 2026-02-21T01:43:36.638 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../78-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-21T01:43:36.638 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T01:43:36.681 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-toml. 2026-02-21T01:43:36.690 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../79-python3-toml_0.10.2-1_all.deb ... 2026-02-21T01:43:36.690 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-21T01:43:36.705 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-pytest. 2026-02-21T01:43:36.714 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../80-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-21T01:43:36.714 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T01:43:36.736 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-simplejson. 2026-02-21T01:43:36.744 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../81-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-21T01:43:36.744 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-21T01:43:36.763 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package python3-webob. 2026-02-21T01:43:36.772 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../82-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-21T01:43:36.772 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T01:43:36.795 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-21T01:43:36.804 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../83-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-21T01:43:36.804 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-21T01:43:36.870 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package radosgw. 2026-02-21T01:43:36.879 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../84-radosgw_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:36.880 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking radosgw (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:37.105 INFO:teuthology.orchestra.run.trial009.stdout:Selecting previously unselected package rbd-fuse. 2026-02-21T01:43:37.114 INFO:teuthology.orchestra.run.trial009.stdout:Preparing to unpack .../85-rbd-fuse_20.2.0-681-g36d388f2-1jammy_amd64.deb ... 2026-02-21T01:43:37.114 INFO:teuthology.orchestra.run.trial009.stdout:Unpacking rbd-fuse (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:37.143 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-21T01:43:37.209 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-21T01:43:37.210 INFO:teuthology.orchestra.run.trial009.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-21T01:43:37.283 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T01:43:37.569 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-21T01:43:37.954 INFO:teuthology.orchestra.run.trial009.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-21T01:43:37.964 INFO:teuthology.orchestra.run.trial009.stdout:Setting up cephadm (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:38.007 INFO:teuthology.orchestra.run.trial009.stdout:Adding system user cephadm....done 2026-02-21T01:43:38.018 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-21T01:43:38.089 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-21T01:43:38.090 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-21T01:43:38.161 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-21T01:43:38.237 INFO:teuthology.orchestra.run.trial009.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-21T01:43:38.237 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-21T01:43:38.337 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-21T01:43:38.461 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-21T01:43:38.535 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-21T01:43:38.611 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-ceph-argparse (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:38.679 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-21T01:43:38.680 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-21T01:43:38.681 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-21T01:43:38.682 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-21T01:43:38.683 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-21T01:43:38.806 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-21T01:43:38.883 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libcephfs-proxy2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:38.884 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-21T01:43:38.963 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-21T01:43:39.051 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-21T01:43:39.309 INFO:teuthology.orchestra.run.trial009.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-21T01:43:39.310 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-21T01:43:39.404 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-21T01:43:39.533 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-21T01:43:39.614 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-21T01:43:39.686 INFO:teuthology.orchestra.run.trial009.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-21T01:43:39.686 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-ceph-common (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:39.775 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-21T01:43:40.290 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T01:43:40.294 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-21T01:43:40.369 INFO:teuthology.orchestra.run.trial009.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-21T01:43:40.370 INFO:teuthology.orchestra.run.trial009.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-21T01:43:40.371 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-21T01:43:40.444 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-21T01:43:40.517 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T01:43:40.518 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-21T01:43:40.601 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-21T01:43:40.679 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-21T01:43:40.759 INFO:teuthology.orchestra.run.trial009.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-21T01:43:40.760 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-21T01:43:40.847 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-21T01:43:40.848 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-21T01:43:40.923 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-21T01:43:41.010 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-21T01:43:41.082 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-21T01:43:41.083 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-21T01:43:41.220 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-21T01:43:41.296 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-21T01:43:41.297 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-21T01:43:41.379 INFO:teuthology.orchestra.run.trial009.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-21T01:43:41.380 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-21T01:43:41.513 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-21T01:43:41.514 INFO:teuthology.orchestra.run.trial009.stdout:Setting up librados2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:41.515 INFO:teuthology.orchestra.run.trial009.stdout:Setting up librgw2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:41.516 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libsqlite3-mod-ceph (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:41.517 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-21T01:43:42.046 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libcephfs2 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.047 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libradosstriper1 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.048 INFO:teuthology.orchestra.run.trial009.stdout:Setting up librbd1 (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.049 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mgr-modules-core (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.050 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-fuse (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.122 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T01:43:42.122 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-21T01:43:42.524 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libcephfs-dev (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.524 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-rados (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.525 INFO:teuthology.orchestra.run.trial009.stdout:Setting up libcephfs-daemon (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.526 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-rbd (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.527 INFO:teuthology.orchestra.run.trial009.stdout:Setting up rbd-fuse (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.528 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-rgw (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.529 INFO:teuthology.orchestra.run.trial009.stdout:Setting up python3-cephfs (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.530 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-common (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:42.557 INFO:teuthology.orchestra.run.trial009.stdout:Adding group ceph....done 2026-02-21T01:43:42.591 INFO:teuthology.orchestra.run.trial009.stdout:Adding system user ceph....done 2026-02-21T01:43:42.601 INFO:teuthology.orchestra.run.trial009.stdout:Setting system user ceph properties....done 2026-02-21T01:43:42.607 INFO:teuthology.orchestra.run.trial009.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-21T01:43:42.694 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-21T01:43:42.978 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-21T01:43:43.369 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-test (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:43.370 INFO:teuthology.orchestra.run.trial009.stdout:Setting up radosgw (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:43.636 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T01:43:43.636 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-21T01:43:44.020 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-base (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:44.099 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-21T01:43:44.472 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mds (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:44.531 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T01:43:44.532 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-21T01:43:44.915 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mgr (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:44.996 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T01:43:44.996 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-21T01:43:45.375 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-osd (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:45.459 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T01:43:45.459 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-21T01:43:45.822 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mgr-k8sevents (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:45.832 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mgr-diskprediction-local (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:45.842 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mon (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:45.913 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T01:43:45.914 INFO:teuthology.orchestra.run.trial009.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-21T01:43:46.308 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mgr-cephadm (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:46.319 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:46.320 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-mgr-dashboard (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:46.331 INFO:teuthology.orchestra.run.trial009.stdout:Setting up ceph-volume (20.2.0-681-g36d388f2-1jammy) ... 2026-02-21T01:43:46.440 INFO:teuthology.orchestra.run.trial009.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-21T01:43:46.525 INFO:teuthology.orchestra.run.trial009.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-21T01:43:47.173 INFO:teuthology.orchestra.run.trial009.stdout: 2026-02-21T01:43:47.173 INFO:teuthology.orchestra.run.trial009.stdout:Running kernel seems to be up-to-date. 2026-02-21T01:43:47.173 INFO:teuthology.orchestra.run.trial009.stdout: 2026-02-21T01:43:47.173 INFO:teuthology.orchestra.run.trial009.stdout:The processor microcode seems to be up-to-date. 2026-02-21T01:43:47.173 INFO:teuthology.orchestra.run.trial009.stdout: 2026-02-21T01:43:47.173 INFO:teuthology.orchestra.run.trial009.stdout:No services need to be restarted. 2026-02-21T01:43:47.174 INFO:teuthology.orchestra.run.trial009.stdout: 2026-02-21T01:43:47.174 INFO:teuthology.orchestra.run.trial009.stdout:No containers need to be restarted. 2026-02-21T01:43:47.174 INFO:teuthology.orchestra.run.trial009.stdout: 2026-02-21T01:43:47.174 INFO:teuthology.orchestra.run.trial009.stdout:No user sessions are running outdated binaries. 2026-02-21T01:43:47.174 INFO:teuthology.orchestra.run.trial009.stdout: 2026-02-21T01:43:47.174 INFO:teuthology.orchestra.run.trial009.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-21T01:43:47.959 INFO:teuthology.orchestra.run.trial009.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-21T01:43:47.966 DEBUG:teuthology.parallel:result is None 2026-02-21T01:43:47.966 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=36d388f27052dd940bdbaf543bb05bdba0acc155 2026-02-21T01:43:48.141 DEBUG:teuthology.orchestra.run.trial009:> dpkg-query -W -f '${Version}' ceph 2026-02-21T01:43:48.151 INFO:teuthology.orchestra.run.trial009.stdout:20.2.0-681-g36d388f2-1jammy 2026-02-21T01:43:48.151 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-681-g36d388f2-1jammy 2026-02-21T01:43:48.151 INFO:teuthology.task.install:The correct ceph version 20.2.0-681-g36d388f2-1jammy is installed. 2026-02-21T01:43:48.152 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T01:43:48.152 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:43:48.152 DEBUG:teuthology.orchestra.run.trial009:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T01:43:48.200 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T01:43:48.200 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:43:48.200 DEBUG:teuthology.orchestra.run.trial009:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T01:43:48.248 DEBUG:teuthology.orchestra.run.trial009:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T01:43:48.294 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T01:43:48.295 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:43:48.295 DEBUG:teuthology.orchestra.run.trial009:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T01:43:48.343 DEBUG:teuthology.orchestra.run.trial009:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T01:43:48.391 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T01:43:48.392 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:43:48.392 DEBUG:teuthology.orchestra.run.trial009:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T01:43:48.439 DEBUG:teuthology.orchestra.run.trial009:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T01:43:48.487 INFO:teuthology.run_tasks:Running task workunit... 2026-02-21T01:43:48.507 INFO:tasks.workunit:Pulling workunits from ref 36d388f27052dd940bdbaf543bb05bdba0acc155 2026-02-21T01:43:48.507 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-21T01:43:48.507 INFO:tasks.workunit:timeout=3h 2026-02-21T01:43:48.507 INFO:tasks.workunit:cleanup=True 2026-02-21T01:43:48.508 DEBUG:teuthology.orchestra.run.trial009:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-21T01:43:48.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T01:43:48.532 INFO:teuthology.orchestra.run.trial009.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-21T01:43:48.532 DEBUG:teuthology.orchestra.run.trial009:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T01:43:48.580 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-21T01:43:48.581 DEBUG:teuthology.orchestra.run.trial009:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-21T01:43:48.628 DEBUG:teuthology.orchestra.run.trial009:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 36d388f27052dd940bdbaf543bb05bdba0acc155 2026-02-21T01:43:48.683 INFO:tasks.workunit.client.0.trial009.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-21T01:44:10.836 INFO:tasks.workunit.client.0.trial009.stderr:Note: switching to '36d388f27052dd940bdbaf543bb05bdba0acc155'. 2026-02-21T01:44:10.836 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.836 INFO:tasks.workunit.client.0.trial009.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-21T01:44:10.836 INFO:tasks.workunit.client.0.trial009.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-21T01:44:10.836 INFO:tasks.workunit.client.0.trial009.stderr:state without impacting any branches by switching back to a branch. 2026-02-21T01:44:10.836 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: git switch -c 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr:Or undo this operation with: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: git switch - 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-21T01:44:10.837 INFO:tasks.workunit.client.0.trial009.stderr: 2026-02-21T01:44:10.838 INFO:tasks.workunit.client.0.trial009.stderr:HEAD is now at 36d388f2705 Merge branch 'aclamk-rocksdb-binned-cache-perf-counters-tentacle' of https://github.com/aclamk/ceph into wip-yuri3-testing-2026-02-19-1610-tentacle 2026-02-21T01:44:10.846 DEBUG:teuthology.orchestra.run.trial009:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-21T01:44:10.894 DEBUG:teuthology.orchestra.run.trial009:> set -ex 2026-02-21T01:44:10.894 DEBUG:teuthology.orchestra.run.trial009:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-21T01:44:10.943 INFO:tasks.workunit:Running workunits matching osd on client.0... 2026-02-21T01:44:10.943 INFO:tasks.workunit:Running workunit osd/bad-inc-map.sh... 2026-02-21T01:44:10.943 DEBUG:teuthology.orchestra.run.trial009:workunit test osd/bad-inc-map.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh 2026-02-21T01:44:10.994 INFO:tasks.workunit.client.0.trial009.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/bad-inc-map 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:8: run: local dir=td/bad-inc-map 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:9: run: shift 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: export CEPH_MON=127.0.0.1:45813 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: CEPH_MON=127.0.0.1:45813 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:12: run: export CEPH_ARGS 2026-02-21T01:44:11.009 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: uuidgen 2026-02-21T01:44:11.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: CEPH_ARGS+='--fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none ' 2026-02-21T01:44:11.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:14: run: CEPH_ARGS+='--mon-host=127.0.0.1:45813 ' 2026-02-21T01:44:11.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:15: run: set -e 2026-02-21T01:44:11.011 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: set 2026-02-21T01:44:11.011 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T01:44:11.014 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: local funcs=TEST_bad_inc_map 2026-02-21T01:44:11.014 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:18: run: for func in $funcs 2026-02-21T01:44:11.014 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:19: run: setup td/bad-inc-map 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/bad-inc-map 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/bad-inc-map 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/bad-inc-map 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/bad-inc-map KILL 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:44:11.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:44:11.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:44:11.016 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:44:11.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:44:11.017 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:44:11.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:44:11.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:44:11.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:44:11.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:11.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:44:11.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:44:11.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:11.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:44:11.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:44:11.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/bad-inc-map 2026-02-21T01:44:11.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:44:11.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12006 2026-02-21T01:44:11.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:44:11.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:44:11.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/bad-inc-map 2026-02-21T01:44:11.025 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:44:11.025 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.025 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.025 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12006 2026-02-21T01:44:11.027 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/bad-inc-map 1' TERM HUP INT 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:20: run: TEST_bad_inc_map td/bad-inc-map 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:26: TEST_bad_inc_map: local dir=td/bad-inc-map 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:28: TEST_bad_inc_map: run_mon td/bad-inc-map a 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/bad-inc-map 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/bad-inc-map/a 2026-02-21T01:44:11.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/bad-inc-map/a --run-dir=td/bad-inc-map 2026-02-21T01:44:11.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:44:11.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:11.061 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:11.061 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:11.061 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.061 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.061 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:11.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/bad-inc-map/a '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --mon-cluster-log-file=td/bad-inc-map/log --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:44:11.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:44:11.094 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:44:11.094 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:44:11.094 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:44:11.094 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:44:11.094 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:44:11.095 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:44:11.095 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:44:11.095 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:44:11.096 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:44:11.096 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.096 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.096 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12006/ceph-mon.a.asok 2026-02-21T01:44:11.096 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:44:11.096 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12006/ceph-mon.a.asok config get fsid 2026-02-21T01:44:11.150 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:44:11.150 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:44:11.150 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.151 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12006/ceph-mon.a.asok 2026-02-21T01:44:11.152 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:44:11.152 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12006/ceph-mon.a.asok config get mon_host 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:29: TEST_bad_inc_map: run_mgr td/bad-inc-map x 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/bad-inc-map 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/bad-inc-map/x 2026-02-21T01:44:11.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:44:11.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:44:11.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:11.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:11.315 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:11.315 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.316 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.316 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:11.316 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:44:11.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/bad-inc-map/x '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:30: TEST_bad_inc_map: run_osd td/bad-inc-map 0 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/bad-inc-map 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/bad-inc-map/0 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 ' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/0' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/0/journal' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:44:11.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:44:11.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/bad-inc-map/0 2026-02-21T01:44:11.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:44:11.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=68fe5adb-3de6-40d2-9182-34084c73f45e 2026-02-21T01:44:11.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 68fe5adb-3de6-40d2-9182-34084c73f45e' 2026-02-21T01:44:11.338 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 68fe5adb-3de6-40d2-9182-34084c73f45e 2026-02-21T01:44:11.338 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:44:11.349 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDrDZlp5je7FBAAUKxG+PbgAkgNxWkUGsQbYQ== 2026-02-21T01:44:11.349 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDrDZlp5je7FBAAUKxG+PbgAkgNxWkUGsQbYQ=="}' 2026-02-21T01:44:11.349 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 68fe5adb-3de6-40d2-9182-34084c73f45e -i td/bad-inc-map/0/new.json 2026-02-21T01:44:11.464 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:11.474 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/bad-inc-map/0/new.json 2026-02-21T01:44:11.475 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDrDZlp5je7FBAAUKxG+PbgAkgNxWkUGsQbYQ== --osd-uuid 68fe5adb-3de6-40d2-9182-34084c73f45e 2026-02-21T01:44:11.491 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:11.484+0000 7f4f7dbb2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:11.492 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:11.488+0000 7f4f7dbb2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:11.493 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:11.488+0000 7f4f7dbb2a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:11.493 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:11.488+0000 7f4f7dbb2a40 -1 bdev(0x55f143086c00 td/bad-inc-map/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:44:11.493 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:11.488+0000 7f4f7dbb2a40 -1 bluestore(td/bad-inc-map/0) _read_fsid unparsable uuid 2026-02-21T01:44:12.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/bad-inc-map/0/keyring 2026-02-21T01:44:12.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:44:12.159 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:44:12.159 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:44:12.159 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/bad-inc-map/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:44:12.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:44:12.273 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:44:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:44:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:44:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:44:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:44:12.288 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:12.280+0000 7f485b5cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:12.289 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:12.284+0000 7f485b5cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:12.291 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:12.284+0000 7f485b5cfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:12.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:12.492 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:12.757 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:12.752+0000 7f485b5cfa40 -1 Falling back to public interface 2026-02-21T01:44:13.493 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:13.493 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:13.493 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:13.494 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:44:13.494 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:13.494 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:13.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:13.835 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:13.828+0000 7f485b5cfa40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:44:14.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:14.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:14.605 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:14.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:44:14.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:14.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:14.810 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2406205518,v1:127.0.0.1:6801/2406205518] [v2:127.0.0.1:6802/2406205518,v1:127.0.0.1:6803/2406205518] exists,up 68fe5adb-3de6-40d2-9182-34084c73f45e 2026-02-21T01:44:14.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:14.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:14.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:14.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:31: TEST_bad_inc_map: run_osd td/bad-inc-map 1 2026-02-21T01:44:14.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/bad-inc-map 2026-02-21T01:44:14.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/bad-inc-map/1 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 ' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/1' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/1/journal' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:14.814 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/bad-inc-map/1 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:44:14.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fa216c8e-7a75-4ec1-a6a8-54c23efee2ab 2026-02-21T01:44:14.816 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 fa216c8e-7a75-4ec1-a6a8-54c23efee2ab 2026-02-21T01:44:14.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 fa216c8e-7a75-4ec1-a6a8-54c23efee2ab' 2026-02-21T01:44:14.816 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:44:14.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDuDZlpgisIMRAAEhe6b8sFgJH4isS9dIs8cg== 2026-02-21T01:44:14.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDuDZlpgisIMRAAEhe6b8sFgJH4isS9dIs8cg=="}' 2026-02-21T01:44:14.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fa216c8e-7a75-4ec1-a6a8-54c23efee2ab -i td/bad-inc-map/1/new.json 2026-02-21T01:44:15.027 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/bad-inc-map/1/new.json 2026-02-21T01:44:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDuDZlpgisIMRAAEhe6b8sFgJH4isS9dIs8cg== --osd-uuid fa216c8e-7a75-4ec1-a6a8-54c23efee2ab 2026-02-21T01:44:15.052 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.048+0000 7f5d856bda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:15.054 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.048+0000 7f5d856bda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:15.055 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.048+0000 7f5d856bda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:15.056 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.048+0000 7f5d856bda40 -1 bdev(0x55bde9362c00 td/bad-inc-map/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:44:15.056 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.048+0000 7f5d856bda40 -1 bluestore(td/bad-inc-map/1) _read_fsid unparsable uuid 2026-02-21T01:44:15.675 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/bad-inc-map/1/keyring 2026-02-21T01:44:15.675 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:44:15.676 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:44:15.676 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:44:15.676 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/bad-inc-map/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:44:15.950 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:44:15.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:44:15.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:44:15.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:44:15.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:44:15.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:44:15.973 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.968+0000 7fd1b8fcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:15.975 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.968+0000 7fd1b8fcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:15.976 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:15.972+0000 7fd1b8fcea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:16.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:44:16.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:16.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:44:16.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:16.164 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:16.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:16.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:16.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:16.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:16.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:44:16.334 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:16.328+0000 7fd1b8fcea40 -1 Falling back to public interface 2026-02-21T01:44:16.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:16.961 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:16.956+0000 7fd1b8fcea40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:44:17.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:17.374 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:17.374 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:44:17.374 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:17.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:44:17.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:17.586 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1504618081,v1:127.0.0.1:6811/1504618081] [v2:127.0.0.1:6812/1504618081,v1:127.0.0.1:6813/1504618081] exists,up fa216c8e-7a75-4ec1-a6a8-54c23efee2ab 2026-02-21T01:44:17.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:17.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:17.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:17.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:32: TEST_bad_inc_map: run_osd td/bad-inc-map 2 2026-02-21T01:44:17.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/bad-inc-map 2026-02-21T01:44:17.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/bad-inc-map/2 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 ' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/2' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/2/journal' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:44:17.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:44:17.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:44:17.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/bad-inc-map/2 2026-02-21T01:44:17.593 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:44:17.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2f3c5754-f7bf-4c69-a27a-ed76facfcfc2 2026-02-21T01:44:17.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 2f3c5754-f7bf-4c69-a27a-ed76facfcfc2' 2026-02-21T01:44:17.593 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 2f3c5754-f7bf-4c69-a27a-ed76facfcfc2 2026-02-21T01:44:17.594 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:44:17.602 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDxDZlpK0zOIxAAKCqRxYInrNO8eoVl6hk1Yg== 2026-02-21T01:44:17.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDxDZlpK0zOIxAAKCqRxYInrNO8eoVl6hk1Yg=="}' 2026-02-21T01:44:17.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2f3c5754-f7bf-4c69-a27a-ed76facfcfc2 -i td/bad-inc-map/2/new.json 2026-02-21T01:44:17.816 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:17.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/bad-inc-map/2/new.json 2026-02-21T01:44:17.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDxDZlpK0zOIxAAKCqRxYInrNO8eoVl6hk1Yg== --osd-uuid 2f3c5754-f7bf-4c69-a27a-ed76facfcfc2 2026-02-21T01:44:17.840 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:17.836+0000 7fd1bc36ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:17.841 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:17.836+0000 7fd1bc36ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:17.842 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:17.836+0000 7fd1bc36ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:17.843 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:17.836+0000 7fd1bc36ba40 -1 bdev(0x562a5ae88c00 td/bad-inc-map/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:44:17.843 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:17.836+0000 7fd1bc36ba40 -1 bluestore(td/bad-inc-map/2) _read_fsid unparsable uuid 2026-02-21T01:44:18.522 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/bad-inc-map/2/keyring 2026-02-21T01:44:18.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:44:18.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:44:18.524 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:44:18.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/bad-inc-map/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:44:18.802 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:44:18.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:44:18.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=67e1d453-dbc6-466a-9b45-ff116d4522ed --auth-supported=none --mon-host=127.0.0.1:45813 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12006/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:44:18.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:44:18.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:44:18.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:44:18.825 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:18.820+0000 7fde52174a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:18.827 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:18.820+0000 7fde52174a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:18.827 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:18.820+0000 7fde52174a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:19.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:44:19.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:19.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:44:19.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:19.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:19.016 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:19.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:19.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:19.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:19.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:19.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:19.238 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:19.232+0000 7fde52174a40 -1 Falling back to public interface 2026-02-21T01:44:19.838 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:19.832+0000 7fde52174a40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:44:20.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:20.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:20.221 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:20.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:44:20.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:20.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:20.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:21.428 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:21.428 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:21.429 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:21.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:44:21.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:21.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:21.631 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1028470774,v1:127.0.0.1:6819/1028470774] [v2:127.0.0.1:6820/1028470774,v1:127.0.0.1:6821/1028470774] exists,up 2f3c5754-f7bf-4c69-a27a-ed76facfcfc2 2026-02-21T01:44:21.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:21.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:21.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:21.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:34: TEST_bad_inc_map: ceph config set osd.2 osd_inject_bad_map_crc_probability 1 2026-02-21T01:44:21.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:37: TEST_bad_inc_map: create_pool foo 8 2026-02-21T01:44:21.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 8 2026-02-21T01:44:22.103 INFO:tasks.workunit.client.0.trial009.stderr:pool 'foo' already exists 2026-02-21T01:44:22.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T01:44:23.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:38: TEST_bad_inc_map: ceph osd pool set foo min_size 1 2026-02-21T01:44:23.416 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 min_size to 1 2026-02-21T01:44:23.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:39: TEST_bad_inc_map: ceph osd pool set foo min_size 2 2026-02-21T01:44:23.721 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 min_size to 2 2026-02-21T01:44:23.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:41: TEST_bad_inc_map: sleep 5 2026-02-21T01:44:28.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: TIMEOUT=10 2026-02-21T01:44:28.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: wait_for_osd up 0 2026-02-21T01:44:28.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:28.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:44:28.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:28.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:28.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 10 )) 2026-02-21T01:44:28.735 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:28.736 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:28.736 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:28.736 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:28.943 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2406205518,v1:127.0.0.1:6801/2406205518] [v2:127.0.0.1:6802/2406205518,v1:127.0.0.1:6803/2406205518] exists,up 68fe5adb-3de6-40d2-9182-34084c73f45e 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: TIMEOUT=10 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: wait_for_osd up 1 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 10 )) 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:28.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:44:29.148 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 9 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1504618081,v1:127.0.0.1:6811/1504618081] [v2:127.0.0.1:6812/1504618081,v1:127.0.0.1:6813/1504618081] exists,up fa216c8e-7a75-4ec1-a6a8-54c23efee2ab 2026-02-21T01:44:29.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:29.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: TIMEOUT=10 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: wait_for_osd up 2 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 10 )) 2026-02-21T01:44:29.150 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:29.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:29.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:29.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:29.354 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1028470774,v1:127.0.0.1:6819/1028470774] [v2:127.0.0.1:6820/1028470774,v1:127.0.0.1:6821/1028470774] exists,up 2f3c5754-f7bf-4c69-a27a-ed76facfcfc2 2026-02-21T01:44:29.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:29.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:29.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:29.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:49: TEST_bad_inc_map: grep 'injecting map crc failure' td/bad-inc-map/osd.2.log 2026-02-21T01:44:29.356 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:22.051+0000 7fde3efac640 -1 osd.2 13 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.356 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:22.151+0000 7fde40fb0640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.356 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:22.151+0000 7fde40fb0640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.356 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.307+0000 7fde40fb0640 -1 osd.2 15 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.357 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.411+0000 7fde40fb0640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.357 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.411+0000 7fde3efac640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.357 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.611+0000 7fde40fb0640 -1 osd.2 18 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.357 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.715+0000 7fde40fb0640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.357 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.715+0000 7fde3efac640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-21T01:44:29.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:50: TEST_bad_inc_map: grep 'bailing because last' td/bad-inc-map/osd.2.log 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:22.051+0000 7fde3efac640 10 osd.2 13 handle_osd_map bailing because last < start (13<14) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:22.151+0000 7fde40fb0640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:22.151+0000 7fde40fb0640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.307+0000 7fde40fb0640 10 osd.2 15 handle_osd_map bailing because last < start (15<16) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.411+0000 7fde40fb0640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.411+0000 7fde3efac640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.611+0000 7fde40fb0640 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.715+0000 7fde40fb0640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-21T01:44:29.358 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:44:23.715+0000 7fde3efac640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-21T01:44:29.359 INFO:tasks.workunit.client.0.trial009.stdout:success 2026-02-21T01:44:29.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:52: TEST_bad_inc_map: echo success 2026-02-21T01:44:29.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:54: TEST_bad_inc_map: delete_pool foo 2026-02-21T01:44:29.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=foo 2026-02-21T01:44:29.360 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete foo foo --yes-i-really-really-mean-it 2026-02-21T01:44:29.589 INFO:tasks.workunit.client.0.trial009.stderr:pool 'foo' does not exist 2026-02-21T01:44:29.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:55: TEST_bad_inc_map: kill_daemons td/bad-inc-map 2026-02-21T01:44:29.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:44:29.600 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:44:29.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:44:29.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:44:29.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:44:34.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:44:34.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:21: run: teardown td/bad-inc-map 2026-02-21T01:44:34.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/bad-inc-map 2026-02-21T01:44:34.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:44:34.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/bad-inc-map KILL 2026-02-21T01:44:34.915 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:44:34.915 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:44:34.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:44:34.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:44:34.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:44:34.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:44:34.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:44:34.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:44:34.923 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:44:34.924 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:44:34.924 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:44:34.925 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:44:34.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:34.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:44:34.927 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:44:34.927 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:34.927 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:44:34.928 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:44:34.929 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/bad-inc-map 2026-02-21T01:44:34.937 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:44:34.937 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:34.937 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:34.938 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12006 2026-02-21T01:44:34.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:44:34.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:44:34.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-21T01:44:34.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/bad-inc-map 0 2026-02-21T01:44:34.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/bad-inc-map 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/bad-inc-map KILL 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:44:34.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:44:34.942 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:44:34.943 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:44:34.943 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:44:34.943 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:44:34.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:44:34.944 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:44:34.944 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:44:34.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:34.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:44:34.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:44:34.946 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:34.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:44:34.947 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T01:44:34.947 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/bad-inc-map 2026-02-21T01:44:34.948 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:44:34.949 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:34.949 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12006 2026-02-21T01:44:34.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12006 2026-02-21T01:44:34.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:44:34.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:44:34.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-21T01:44:34.951 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T01:44:34.952 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T01:44:35.000 INFO:tasks.workunit:Running workunit osd/divergent-priors.sh... 2026-02-21T01:44:35.001 DEBUG:teuthology.orchestra.run.trial009:workunit test osd/divergent-priors.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh 2026-02-21T01:44:35.053 INFO:tasks.workunit.client.0.trial009.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T01:44:35.057 INFO:tasks.workunit.client.0.trial009.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T01:44:35.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:44:35.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:44:35.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/divergent-priors 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:21: run: local dir=td/divergent-priors 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:22: run: shift 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: export loglen=12 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: loglen=12 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: export divisor=3 2026-02-21T01:44:35.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: divisor=3 2026-02-21T01:44:35.059 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: expr 12 / 2 2026-02-21T01:44:35.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: export trim=6 2026-02-21T01:44:35.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: trim=6 2026-02-21T01:44:35.059 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: expr 6 / 3 2026-02-21T01:44:35.060 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: export DIVERGENT_WRITE=2 2026-02-21T01:44:35.060 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: DIVERGENT_WRITE=2 2026-02-21T01:44:35.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: expr 6 / 3 2026-02-21T01:44:35.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: export DIVERGENT_REMOVE=2 2026-02-21T01:44:35.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: DIVERGENT_REMOVE=2 2026-02-21T01:44:35.062 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: expr 6 / 3 2026-02-21T01:44:35.063 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: export DIVERGENT_CREATE=2 2026-02-21T01:44:35.063 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: DIVERGENT_CREATE=2 2026-02-21T01:44:35.063 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: export poolname=test 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: poolname=test 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: export testobjects=100 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: testobjects=100 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: export CEPH_MON=127.0.0.1:7115 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: CEPH_MON=127.0.0.1:7115 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:35: run: export CEPH_ARGS 2026-02-21T01:44:35.064 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: uuidgen 2026-02-21T01:44:35.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: CEPH_ARGS+='--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none ' 2026-02-21T01:44:35.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:37: run: CEPH_ARGS+='--mon-host=127.0.0.1:7115 ' 2026-02-21T01:44:35.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:39: run: CEPH_ARGS+='--osd_force_auth_primary_missing_objects=1000000 ' 2026-02-21T01:44:35.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:40: run: CEPH_ARGS+='--osd_debug_pg_log_writeout=true ' 2026-02-21T01:44:35.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:41: run: CEPH_ARGS+='--osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:44:35.066 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: set 2026-02-21T01:44:35.066 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T01:44:35.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: local 'funcs=TEST_divergent 2026-02-21T01:44:35.068 INFO:tasks.workunit.client.0.trial009.stderr:TEST_divergent_2 2026-02-21T01:44:35.068 INFO:tasks.workunit.client.0.trial009.stderr:TEST_divergent_3 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:TEST_divergent_ec' 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:44:35.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:44:35.070 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:44:35.070 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:44:35.071 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:44:35.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:44:35.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:44:35.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:44:35.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:44:35.073 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:44:35.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:35.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:44:35.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:44:35.074 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:44:35.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:44:35.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:44:35.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:44:35.079 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:44:35.079 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.079 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:44:35.080 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:44:35.080 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:44:35.080 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-21T01:44:35.082 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:44:35.082 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.082 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.20522 2026-02-21T01:44:35.083 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:44:35.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T01:44:35.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T01:44:35.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:44:35.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-21T01:44:35.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent td/divergent-priors 2026-02-21T01:44:35.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:57: TEST_divergent: local dir=td/divergent-priors 2026-02-21T01:44:35.086 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:59: TEST_divergent: file_with_random_data 2026-02-21T01:44:35.086 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=512 2026-02-21T01:44:35.086 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:44:35.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.ussY4xfZwU 2026-02-21T01:44:35.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.ussY4xfZwU bs=512 count=1 2026-02-21T01:44:35.088 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:44:35.088 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:44:35.088 INFO:tasks.workunit.client.0.trial009.stderr:512 bytes copied, 8.9727e-05 s, 5.7 MB/s 2026-02-21T01:44:35.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:35.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:59: TEST_divergent: local dummyfile=/tmp/tmp.ussY4xfZwU 2026-02-21T01:44:35.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:60: TEST_divergent: local num_osds=3 2026-02-21T01:44:35.089 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: expr 3 - 1 2026-02-21T01:44:35.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: seq 0 2 2026-02-21T01:44:35.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: local 'osds=0 2026-02-21T01:44:35.091 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:44:35.091 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:44:35.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:62: TEST_divergent: run_mon td/divergent-priors a 2026-02-21T01:44:35.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-21T01:44:35.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:44:35.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:44:35.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:44:35.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-21T01:44:35.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:35.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:44:35.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:44:35.155 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:44:35.155 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:44:35.155 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:44:35.155 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:44:35.155 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:44:35.157 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:44:35.157 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:44:35.157 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:44:35.157 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:44:35.157 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.157 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.158 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:44:35.158 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:44:35.158 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get fsid 2026-02-21T01:44:35.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:44:35.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:44:35.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:44:35.222 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.223 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.223 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:44:35.223 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:44:35.223 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get mon_host 2026-02-21T01:44:35.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:63: TEST_divergent: run_mgr td/divergent-priors x 2026-02-21T01:44:35.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-21T01:44:35.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:44:35.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:44:35.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:44:35.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-21T01:44:35.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:44:35.394 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:44:35.394 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:35.394 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:35.394 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:35.394 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.394 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.395 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:35.395 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:44:35.395 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: for i in $osds 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_osd td/divergent-priors 0 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:44:35.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:44:35.417 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:44:35.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:44:35.419 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:44:35.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=831cb369-2a0c-4f59-9212-c85d454f3cab 2026-02-21T01:44:35.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 831cb369-2a0c-4f59-9212-c85d454f3cab' 2026-02-21T01:44:35.421 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 831cb369-2a0c-4f59-9212-c85d454f3cab 2026-02-21T01:44:35.421 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:44:35.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQADDplpxGuPGRAA5ffza3CJxjYLB7duXzXPNg== 2026-02-21T01:44:35.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQADDplpxGuPGRAA5ffza3CJxjYLB7duXzXPNg=="}' 2026-02-21T01:44:35.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 831cb369-2a0c-4f59-9212-c85d454f3cab -i td/divergent-priors/0/new.json 2026-02-21T01:44:35.543 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:35.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-21T01:44:35.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQADDplpxGuPGRAA5ffza3CJxjYLB7duXzXPNg== --osd-uuid 831cb369-2a0c-4f59-9212-c85d454f3cab 2026-02-21T01:44:35.570 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:35.563+0000 7f7e776fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:35.572 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:35.563+0000 7f7e776fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:35.573 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:35.567+0000 7f7e776fba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:35.573 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:35.567+0000 7f7e776fba40 -1 bdev(0x56471dcb4c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:44:35.573 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:35.567+0000 7f7e776fba40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-21T01:44:36.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-21T01:44:36.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:44:36.317 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:44:36.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:44:36.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:44:36.443 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:44:36.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:44:36.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:44:36.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:44:36.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:44:36.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:44:36.476 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:36.471+0000 7fdcee866a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:36.479 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:36.471+0000 7fdcee866a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:36.480 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:36.475+0000 7fdcee866a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:36.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:36.555 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:36.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:36.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:36.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:37.046 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:37.039+0000 7fdcee866a40 -1 Falling back to public interface 2026-02-21T01:44:37.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:37.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:37.668 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:37.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:44:37.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:37.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:37.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:38.039 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:38.031+0000 7fdcee866a40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:44:38.882 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:38.882 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:38.883 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:38.884 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:44:38.884 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:38.884 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:44:39.097 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2339919842,v1:127.0.0.1:6801/2339919842] [v2:127.0.0.1:6802/2339919842,v1:127.0.0.1:6803/2339919842] exists,up 831cb369-2a0c-4f59-9212-c85d454f3cab 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: for i in $osds 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_osd td/divergent-priors 1 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:39.098 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:44:39.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:44:39.100 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:44:39.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2936fbec-cae3-4d07-8448-09e5c05cd5b1 2026-02-21T01:44:39.101 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 2936fbec-cae3-4d07-8448-09e5c05cd5b1 2026-02-21T01:44:39.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2936fbec-cae3-4d07-8448-09e5c05cd5b1' 2026-02-21T01:44:39.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:44:39.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAHDplp0EPOBhAA9pKQ5X/6HpF8RLffJJP58Q== 2026-02-21T01:44:39.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAHDplp0EPOBhAA9pKQ5X/6HpF8RLffJJP58Q=="}' 2026-02-21T01:44:39.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2936fbec-cae3-4d07-8448-09e5c05cd5b1 -i td/divergent-priors/1/new.json 2026-02-21T01:44:39.335 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:39.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-21T01:44:39.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAHDplp0EPOBhAA9pKQ5X/6HpF8RLffJJP58Q== --osd-uuid 2936fbec-cae3-4d07-8448-09e5c05cd5b1 2026-02-21T01:44:39.366 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:39.359+0000 7f008ce3ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:39.367 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:39.359+0000 7f008ce3ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:39.368 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:39.363+0000 7f008ce3ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:39.369 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:39.363+0000 7f008ce3ba40 -1 bdev(0x56492bc46c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:44:39.369 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:39.363+0000 7f008ce3ba40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-21T01:44:40.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-21T01:44:40.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:44:40.028 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:44:40.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:44:40.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:44:40.329 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:44:40.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:44:40.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:44:40.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:44:40.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:44:40.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:44:40.358 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:40.351+0000 7f115927ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:40.360 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:40.355+0000 7f115927ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:40.361 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:40.355+0000 7f115927ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:40.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:44:40.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:40.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:44:40.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:40.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:40.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:40.545 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:40.545 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:40.545 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:40.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:44:40.759 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:40.850 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:40.843+0000 7f115927ea40 -1 Falling back to public interface 2026-02-21T01:44:41.514 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:41.507+0000 7f115927ea40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:44:41.760 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:41.761 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:41.761 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:41.762 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:44:41.762 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:41.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:44:41.968 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:42.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:42.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:42.971 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:42.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:44:42.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:42.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:44:43.184 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2672200289,v1:127.0.0.1:6811/2672200289] [v2:127.0.0.1:6812/2672200289,v1:127.0.0.1:6813/2672200289] exists,up 2936fbec-cae3-4d07-8448-09e5c05cd5b1 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: for i in $osds 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_osd td/divergent-priors 2 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:44:43.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:44:43.189 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:44:43.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:44:43.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:44:43.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:44:43.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:44:43.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:44:43.191 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:44:43.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c42ad328-be10-4fdf-8f0d-26b719fe5157 2026-02-21T01:44:43.192 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 c42ad328-be10-4fdf-8f0d-26b719fe5157' 2026-02-21T01:44:43.192 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 c42ad328-be10-4fdf-8f0d-26b719fe5157 2026-02-21T01:44:43.192 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:44:43.203 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQALDplpZ6gBDBAAUg/v4KE6coaHINjODzTOvA== 2026-02-21T01:44:43.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQALDplpZ6gBDBAAUg/v4KE6coaHINjODzTOvA=="}' 2026-02-21T01:44:43.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c42ad328-be10-4fdf-8f0d-26b719fe5157 -i td/divergent-priors/2/new.json 2026-02-21T01:44:43.431 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:43.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-21T01:44:43.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQALDplpZ6gBDBAAUg/v4KE6coaHINjODzTOvA== --osd-uuid c42ad328-be10-4fdf-8f0d-26b719fe5157 2026-02-21T01:44:43.461 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:43.455+0000 7f8f0d130a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:43.463 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:43.455+0000 7f8f0d130a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:43.464 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:43.455+0000 7f8f0d130a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:43.464 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:43.459+0000 7f8f0d130a40 -1 bdev(0x5562cb9fcc00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:44:43.464 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:43.459+0000 7f8f0d130a40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-21T01:44:44.095 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-21T01:44:44.095 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:44:44.096 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:44:44.096 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:44:44.096 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:44:44.391 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:44:44.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:44:44.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:44:44.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:44:44.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:44:44.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:44:44.410 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:44.403+0000 7ffbe4d7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:44.411 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:44.403+0000 7ffbe4d7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:44.412 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:44.407+0000 7ffbe4d7fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:44:44.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:44:44.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:44:44.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:44:44.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:44:44.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:44:44.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:44.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:44:44.600 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:44:44.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:44.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:44.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:44.882 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:44.875+0000 7ffbe4d7fa40 -1 Falling back to public interface 2026-02-21T01:44:45.450 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:44:45.443+0000 7ffbe4d7fa40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:44:45.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:45.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:45.820 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:44:45.821 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:44:45.821 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:45.821 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:46.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:44:47.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:44:47.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:44:47.029 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:44:47.030 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:44:47.030 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:44:47.030 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:44:47.237 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2043228714,v1:127.0.0.1:6819/2043228714] [v2:127.0.0.1:6820/2043228714,v1:127.0.0.1:6821/2043228714] exists,up c42ad328-be10-4fdf-8f0d-26b719fe5157 2026-02-21T01:44:47.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:44:47.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:44:47.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:44:47.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: ceph osd set noout 2026-02-21T01:44:47.544 INFO:tasks.workunit.client.0.trial009.stderr:noout is set 2026-02-21T01:44:47.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:70: TEST_divergent: ceph osd set noin 2026-02-21T01:44:47.849 INFO:tasks.workunit.client.0.trial009.stderr:noin is set 2026-02-21T01:44:47.860 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:71: TEST_divergent: ceph osd set nodown 2026-02-21T01:44:48.152 INFO:tasks.workunit.client.0.trial009.stderr:nodown is set 2026-02-21T01:44:48.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:72: TEST_divergent: create_pool test 1 1 2026-02-21T01:44:48.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-21T01:44:48.406 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' already exists 2026-02-21T01:44:48.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T01:44:49.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:73: TEST_divergent: ceph osd pool set test size 3 2026-02-21T01:44:49.723 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 size to 3 2026-02-21T01:44:49.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:74: TEST_divergent: ceph osd pool set test min_size 2 2026-02-21T01:44:50.028 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 min_size to 2 2026-02-21T01:44:50.039 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:76: TEST_divergent: flush_pg_stats 2026-02-21T01:44:50.039 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:44:50.039 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:44:50.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:44:50.250 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:44:50.250 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:44:50.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:44:50.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:50.250 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:44:50.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-21T01:44:50.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-21T01:44:50.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-21T01:44:50.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:50.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:44:50.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-21T01:44:50.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-21T01:44:50.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-21T01:44:50.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:50.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:44:50.496 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-21T01:44:50.496 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-21T01:44:50.496 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-55834574851' 2026-02-21T01:44:50.496 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:50.496 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-21T01:44:50.496 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:50.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:44:50.498 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-21T01:44:50.499 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:50.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-21T01:44:50.500 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836485 2026-02-21T01:44:50.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-21T01:44:50.500 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:44:50.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-21T01:44:50.721 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:44:51.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:44:51.722 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:44:51.931 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-21T01:44:51.931 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:51.932 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-21T01:44:51.932 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:51.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:44:51.934 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-21T01:44:51.934 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:51.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-21T01:44:51.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-21T01:44:51.935 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705668 2026-02-21T01:44:51.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:44:52.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-21T01:44:52.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:52.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-21T01:44:52.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:52.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:44:52.164 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574851 2026-02-21T01:44:52.164 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:52.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574851 2026-02-21T01:44:52.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-21T01:44:52.166 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574851 2026-02-21T01:44:52.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:44:52.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-21T01:44:52.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:77: TEST_divergent: wait_for_clean 2026-02-21T01:44:52.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:44:52.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:44:52.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:44:52.397 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:44:52.398 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:44:52.398 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:44:52.398 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:44:52.398 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:44:52.398 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:44:52.494 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:44:52.494 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:44:52.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:44:52.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:44:52.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:44:52.495 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:44:52.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:44:52.712 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:44:52.712 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:44:52.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:44:52.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:52.712 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:44:52.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-21T01:44:52.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-21T01:44:52.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-21T01:44:52.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:52.797 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:44:52.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705671 2026-02-21T01:44:52.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705671 2026-02-21T01:44:52.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705671' 2026-02-21T01:44:52.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:52.876 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:44:52.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-21T01:44:52.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-21T01:44:52.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705671 2-55834574854' 2026-02-21T01:44:52.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:52.965 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-21T01:44:52.965 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:52.966 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:44:52.967 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-21T01:44:52.967 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:52.968 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-21T01:44:52.968 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-21T01:44:52.968 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836487 2026-02-21T01:44:52.969 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:44:53.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-21T01:44:53.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:44:54.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:44:54.182 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:44:54.402 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-21T01:44:54.402 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:54.403 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705671 2026-02-21T01:44:54.403 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:54.405 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:44:54.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705671 2026-02-21T01:44:54.406 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:54.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705671 2026-02-21T01:44:54.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-21T01:44:54.407 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705671 2026-02-21T01:44:54.408 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:44:54.626 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-21T01:44:54.626 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:54.627 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574854 2026-02-21T01:44:54.627 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:54.628 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:44:54.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574854 2026-02-21T01:44:54.629 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:54.630 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574854 2026-02-21T01:44:54.630 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574854 2026-02-21T01:44:54.631 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-21T01:44:54.631 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:44:54.856 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-21T01:44:54.856 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:44:54.857 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:44:54.857 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:44:55.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:44:55.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:44:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:44:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:44:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:44:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:44:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:44:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:44:55.366 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:44:55.367 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:44:55.367 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:44:55.368 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:44:55.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:44:55.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:44:55.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:44:55.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: ceph pg dump pgs --format=json 2026-02-21T01:44:55.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: jq '.pg_stats[0].up_primary' 2026-02-21T01:44:55.860 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:44:55.870 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: local divergent=1 2026-02-21T01:44:55.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:81: TEST_divergent: echo 'primary and soon to be divergent is 1' 2026-02-21T01:44:55.871 INFO:tasks.workunit.client.0.trial009.stdout:primary and soon to be divergent is 1 2026-02-21T01:44:55.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:82: TEST_divergent: ceph pg dump pgs 2026-02-21T01:44:56.069 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:44:56.069 INFO:tasks.workunit.client.0.trial009.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T01:44:48.510794+0000 0'0 26:24 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:08:49.351320+0000 0 0 2026-02-21T01:44:56.069 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:44:56.069 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:44:56.070 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:44:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: local non_divergent= 2026-02-21T01:44:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: for i in $osds 2026-02-21T01:44:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: '[' 0 = 1 ']' 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: non_divergent=' 0' 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: for i in $osds 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: '[' 1 = 1 ']' 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: continue 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: for i in $osds 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: '[' 2 = 1 ']' 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: non_divergent=' 0 2' 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: echo 'writing initial objects' 2026-02-21T01:44:56.079 INFO:tasks.workunit.client.0.trial009.stdout:writing initial objects 2026-02-21T01:44:56.080 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: seq 1 100 2026-02-21T01:44:56.080 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.080 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_1 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_2 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_3 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.156 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.156 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_4 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_5 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_6 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_7 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.266 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.266 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_8 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.293 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_9 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.322 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.322 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_10 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_11 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_12 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.405 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.406 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_13 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.433 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.433 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_14 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.461 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.461 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_15 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_16 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_17 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_18 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_19 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_20 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_21 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_22 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_23 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_24 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_25 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.765 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_26 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.793 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.793 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_27 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.821 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.821 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_28 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.849 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.849 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_29 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.877 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.877 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_30 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.905 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.905 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_31 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_32 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_33 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:56.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:56.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_34 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_35 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_36 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_37 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.102 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_38 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_39 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.157 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.157 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_40 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.185 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.185 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_41 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_42 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.234 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.235 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_43 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.262 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_44 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_45 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_46 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_47 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_48 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.400 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.400 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_49 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_50 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_51 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.484 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.484 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_52 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_53 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_54 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_55 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_56 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_57 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_58 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_59 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.702 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.702 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_60 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.729 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.730 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_61 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_62 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.785 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_63 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_64 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_65 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.868 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.868 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_66 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_67 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_68 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_69 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.966 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.966 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_70 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:57.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:57.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_71 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_72 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_73 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_74 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.096 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.097 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_75 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_76 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_77 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_78 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_79 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_80 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_81 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_82 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_83 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_84 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_85 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.401 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.401 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_86 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_87 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_88 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_89 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.514 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.514 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_90 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.542 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.542 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_91 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.570 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.570 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_92 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_93 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.626 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_94 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.654 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.654 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_95 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.674 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.674 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_96 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.702 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.702 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_97 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.730 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_98 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.759 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.759 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_99 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.787 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:94: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-21T01:44:58.787 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:96: TEST_divergent: rados -p test put existing_100 /tmp/tmp.ussY4xfZwU 2026-02-21T01:44:58.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:44:58.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: wait_for_clean 2026-02-21T01:44:58.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:44:58.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:44:58.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:44:58.816 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:44:58.817 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:44:58.817 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:44:58.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:44:58.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:44:58.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:44:58.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:44:58.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:44:58.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:44:58.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:44:58.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:44:58.885 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:44:59.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:44:59.092 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:44:59.093 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:44:59.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:44:59.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:59.093 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:44:59.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836490 2026-02-21T01:44:59.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836490 2026-02-21T01:44:59.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490' 2026-02-21T01:44:59.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:59.177 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:44:59.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-21T01:44:59.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-21T01:44:59.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-38654705674' 2026-02-21T01:44:59.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:44:59.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:44:59.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-21T01:44:59.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-21T01:44:59.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836490 1-38654705674 2-55834574857' 2026-02-21T01:44:59.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:44:59.341 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836490 2026-02-21T01:44:59.341 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:44:59.342 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:44:59.342 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836490 2026-02-21T01:44:59.343 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:44:59.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836490 2026-02-21T01:44:59.344 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836490 2026-02-21T01:44:59.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836490' 2026-02-21T01:44:59.345 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:44:59.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836490 2026-02-21T01:44:59.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:45:00.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:45:00.552 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:45:00.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836490 2026-02-21T01:45:00.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:45:00.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-21T01:45:00.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:45:00.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:45:00.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705674 2026-02-21T01:45:00.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:45:00.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705674 2026-02-21T01:45:00.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-21T01:45:00.777 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705674 2026-02-21T01:45:00.778 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:45:00.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-21T01:45:00.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:45:00.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574857 2026-02-21T01:45:00.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:45:00.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:45:00.991 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574857 2026-02-21T01:45:00.991 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:45:00.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-21T01:45:00.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-21T01:45:00.993 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574857 2026-02-21T01:45:00.993 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:45:01.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-21T01:45:01.216 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:45:01.217 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:01.217 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:01.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:45:01.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:01.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:01.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:01.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:01.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:01.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:01.511 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:01.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:45:01.720 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:01.721 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:01.721 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:01.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:45:01.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:45:01.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:45:01.999 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:101: TEST_divergent: get_pg test existing_1 2026-02-21T01:45:01.999 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-21T01:45:01.999 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-21T01:45:01.999 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-21T01:45:01.999 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-21T01:45:02.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:101: TEST_divergent: local pgid=1.0 2026-02-21T01:45:02.212 INFO:tasks.workunit.client.0.trial009.stdout:blackholing osds 0 2 2026-02-21T01:45:02.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: echo 'blackholing osds 0 2' 2026-02-21T01:45:02.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:105: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:02.417 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:02.417 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-21T01:44:48.510794+0000 26'100 26:124 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:08:49.351320+0000 0 0 2026-02-21T01:45:02.417 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:02.417 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:02.417 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:02.425 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:106: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: get_asok_path osd.0 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:02.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:45:02.428 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: CEPH_ARGS= 2026-02-21T01:45:02.428 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-21T01:45:02.482 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:45:02.482 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:45:02.482 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:45:02.488 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:106: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:02.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: get_asok_path osd.2 2026-02-21T01:45:02.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:45:02.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:45:02.489 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:02.489 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:02.490 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:02.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:45:02.490 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: CEPH_ARGS= 2026-02-21T01:45:02.490 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-21T01:45:02.544 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:45:02.544 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:45:02.544 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:45:02.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: local case5=100 2026-02-21T01:45:02.550 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:112: TEST_divergent: expr 100 - 1 2026-02-21T01:45:02.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:112: TEST_divergent: local case3=99 2026-02-21T01:45:02.551 INFO:tasks.workunit.client.0.trial009.stdout:writing divergent object 2026-02-21T01:45:02.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:114: TEST_divergent: echo 'writing divergent object' 2026-02-21T01:45:02.551 INFO:tasks.workunit.client.0.trial009.stdout:create missing divergent object 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:116: TEST_divergent: echo 'create missing divergent object' 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:117: TEST_divergent: inject_eio rep data test existing_99 td/divergent-priors 0 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=rep 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=test 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=existing_99 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/divergent-priors 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: rados -p test put existing_100 /tmp/tmp.ussY4xfZwU 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds test existing_99 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=test 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=existing_99 2026-02-21T01:45:02.552 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map test existing_99 2026-02-21T01:45:02.553 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=1 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr:0 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 1 0 2 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('1' '0' '2') 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-21T01:45:02.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-21T01:45:02.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' rep '!=' ec ']' 2026-02-21T01:45:02.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2497: inject_eio: shard_id= 2026-02-21T01:45:02.771 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/divergent-priors/1/type 2026-02-21T01:45:02.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-21T01:45:02.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-21T01:45:02.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-21T01:45:02.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-21T01:45:02.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-21T01:45:02.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-21T01:45:02.773 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-21T01:45:02.773 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-21T01:45:02.773 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:45:02.773 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:45:02.775 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:02.775 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:02.775 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:02.775 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:45:02.775 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.20522/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-21T01:45:02.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-21T01:45:02.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-21T01:45:02.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-21T01:45:02.835 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-21T01:45:02.835 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:45:02.835 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:45:02.836 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:02.836 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:02.836 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:02.836 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:45:02.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-21T01:45:02.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok injectdataerr test existing_99 2026-02-21T01:45:02.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:119: TEST_divergent: sleep 10 2026-02-21T01:45:02.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:118: TEST_divergent: rados -p test get existing_99 td/divergent-priors/existing 2026-02-21T01:45:12.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:120: TEST_divergent: killall -9 rados 2026-02-21T01:45:12.942 INFO:tasks.workunit.client.0.trial009.stderr:rados: no process found 2026-02-21T01:45:12.942 INFO:tasks.workunit.client.0.trial009.stdout:killing all the osds 2026-02-21T01:45:12.942 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:123: TEST_divergent: echo 'killing all the osds' 2026-02-21T01:45:12.942 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:124: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:13.161 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:13.161 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51200 0 0 17 84 17 active+clean 2026-02-21T01:45:02.914381+0000 26'101 26:133 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:08:49.351320+0000 0 0 2026-02-21T01:45:13.161 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:13.162 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:13.162 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:13.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:125: TEST_divergent: kill_daemons td/divergent-priors KILL osd 2026-02-21T01:45:13.171 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:45:13.171 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:45:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:45:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:45:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:45:13.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:45:13.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: for i in $osds 2026-02-21T01:45:13.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: ceph osd down osd.0 2026-02-21T01:45:13.750 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:45:13.761 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: for i in $osds 2026-02-21T01:45:13.762 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: ceph osd down osd.1 2026-02-21T01:45:14.025 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:45:14.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: for i in $osds 2026-02-21T01:45:14.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: ceph osd down osd.2 2026-02-21T01:45:14.312 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:45:14.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:130: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:14.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:132: TEST_divergent: ceph osd out osd.0 2026-02-21T01:45:14.614 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already out. 2026-02-21T01:45:14.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:130: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:14.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:132: TEST_divergent: ceph osd out osd.2 2026-02-21T01:45:14.899 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already out. 2026-02-21T01:45:14.908 INFO:tasks.workunit.client.0.trial009.stdout:bringing up non_divergent 0 2 2026-02-21T01:45:14.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:136: TEST_divergent: echo 'bringing up non_divergent 0 2' 2026-02-21T01:45:14.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:137: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:15.118 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:15.118 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51200 0 0 17 84 17 stale+active+clean 2026-02-21T01:45:02.914381+0000 26'101 26:133 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T05:08:49.351320+0000 0 0 2026-02-21T01:45:15.118 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:15.118 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:15.119 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:15.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:138: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: activate_osd td/divergent-priors 0 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:45:15.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:15.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:45:15.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:45:15.133 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:45:15.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:45:15.157 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:15.155+0000 7f65b33afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:15.158 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:15.155+0000 7f65b33afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:15.159 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:15.155+0000 7f65b33afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:15.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:45:15.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:45:15.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:45:15.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:45:15.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:45:15.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:15.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:45:15.357 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:45:15.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:15.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:45:15.537 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:15.535+0000 7f65b33afa40 -1 Falling back to public interface 2026-02-21T01:45:15.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:45:16.220 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:16.215+0000 7f65b33afa40 -1 osd.0 26 log_to_monitors true 2026-02-21T01:45:16.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:45:16.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:16.578 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:45:16.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:45:16.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:16.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:45:16.791 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up out weight 0 up_from 33 up_thru 0 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6800/1331655318,v1:127.0.0.1:6801/1331655318] [v2:127.0.0.1:6802/1331655318,v1:127.0.0.1:6803/1331655318] exists,up 831cb369-2a0c-4f59-9212-c85d454f3cab 2026-02-21T01:45:16.794 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:45:16.794 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:138: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: activate_osd td/divergent-priors 2 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:45:16.795 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:45:16.796 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:45:16.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:45:16.798 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:45:16.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:45:16.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:45:16.800 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-21T01:45:16.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:45:16.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:45:16.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:45:16.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:45:16.820 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:16.819+0000 7f88e74c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:16.822 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:16.819+0000 7f88e74c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:16.823 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:16.819+0000 7f88e74c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:45:17.011 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:45:17.012 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:17.012 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:45:17.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:45:17.294 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:17.291+0000 7f88e74c0a40 -1 Falling back to public interface 2026-02-21T01:45:17.937 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:17.935+0000 7f88e74c0a40 -1 osd.2 26 log_to_monitors true 2026-02-21T01:45:18.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:45:18.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:18.222 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:45:18.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:45:18.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:18.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:45:18.440 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/1539761431,v1:127.0.0.1:6811/1539761431] [v2:127.0.0.1:6812/1539761431,v1:127.0.0.1:6813/1539761431] exists,up c42ad328-be10-4fdf-8f0d-26b719fe5157 2026-02-21T01:45:18.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:45:18.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:45:18.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:45:18.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:142: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:18.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:144: TEST_divergent: ceph osd in osd.0 2026-02-21T01:45:18.682 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already in. 2026-02-21T01:45:18.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:142: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:18.692 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:144: TEST_divergent: ceph osd in osd.2 2026-02-21T01:45:18.962 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already in. 2026-02-21T01:45:18.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:45:18.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: wait_for_clean 2026-02-21T01:45:18.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:45:18.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:45:18.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:45:18.972 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:45:18.973 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:45:18.973 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:45:18.973 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:45:18.974 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:45:18.974 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:45:19.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:45:19.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:45:19.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:45:19.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:45:19.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:45:19.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:45:19.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:45:19.251 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:45:19.251 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:45:19.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:45:19.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:45:19.251 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:45:19.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920771 2026-02-21T01:45:19.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920771 2026-02-21T01:45:19.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-141733920771' 2026-02-21T01:45:19.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:45:19.333 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:45:19.390 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:45:19.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:45:19.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:45:19.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:45:19.394 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:45:19.394 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:45:19.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855363 2026-02-21T01:45:19.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855363 2026-02-21T01:45:19.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-141733920771 2-150323855363' 2026-02-21T01:45:19.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:45:19.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-141733920771 2026-02-21T01:45:19.478 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:45:19.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:45:19.479 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-141733920771 2026-02-21T01:45:19.479 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:45:19.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920771 2026-02-21T01:45:19.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 141733920771' 2026-02-21T01:45:19.481 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 141733920771 2026-02-21T01:45:19.481 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:45:19.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920771 -lt 141733920771 2026-02-21T01:45:19.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:45:19.688 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-150323855363 2026-02-21T01:45:19.688 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:45:19.689 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:45:19.689 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-150323855363 2026-02-21T01:45:19.690 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:45:19.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855363 2026-02-21T01:45:19.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 150323855363' 2026-02-21T01:45:19.691 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 150323855363 2026-02-21T01:45:19.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:45:19.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-21T01:45:19.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:45:19.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:19.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:20.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:20.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:20.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:20.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:20.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:20.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:20.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:45:20.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:45:20.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:45:20.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:20.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:45:20.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:20.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:20.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:20.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:20.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:20.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:20.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:20.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:21.039 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:21.040 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:21.040 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:21.040 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:21.335 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:21.624 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:21.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:21.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:45:21.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:21.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:45:21.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:21.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:21.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:21.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:21.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:21.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:21.826 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:21.826 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:22.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:22.043 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:22.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:22.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:22.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:22.335 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:22.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:22.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:22.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:45:22.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:22.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:45:23.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:23.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:23.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:23.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:23.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:23.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:23.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:23.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:23.226 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:23.227 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:23.227 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:23.227 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:23.501 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:23.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:23.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:23.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:45:23.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:23.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:45:24.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:24.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:24.577 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:24.577 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:24.577 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:24.577 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:24.577 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:24.578 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:24.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:24.786 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:24.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:24.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:25.059 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:25.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:25.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:25.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:45:25.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:25.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:45:26.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:26.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:26.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:26.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:26.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:26.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:26.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:26.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:27.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:27.149 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:27.149 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:27.149 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:27.439 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:27.440 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:27.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:27.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:27.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:45:27.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:27.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:45:30.925 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:30.925 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:30.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:30.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:30.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:30.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:30.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:30.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:31.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:31.139 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:31.139 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:31.139 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:31.427 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:31.428 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:31.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:31.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:31.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:45:31.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:31.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:45:38.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:38.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:38.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:38.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:38.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:38.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:38.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:38.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:38.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:38.327 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:38.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:38.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:38.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:38.618 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:38.618 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:38.902 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:38.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:38.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:45:38.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:45:38.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:45:46.203 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:45:46.203 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:45:46.204 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:45:46.205 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:45:46.205 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:45:46.205 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:45:46.205 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:45:46.205 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:45:46.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:45:46.415 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:45:46.415 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:45:46.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:45:46.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:45:46.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:45:46.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:45:46.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:45:46.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:45:47.251 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:45:47.251 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "026f157c-d8d6-4869-a07f-67457db613cc", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:45:47.249323+0000", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:45:47.252 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.253 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.254 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:45:47.255 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:44:35.110979Z", 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:44:35.110979Z", 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:45:47.256 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:45:47.257 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.258 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.259 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 40, 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:44:35.196460+0000", 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:45:26.352689+0000", 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:45:18.090365+0000", 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:45:18.906263+0000", 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:45:47.260 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 9, 2026-02-21T01:45:47.261 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.262 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:44:48.326756+0000", 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 1, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool", 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 0, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:45:47.263 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "off", 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 1, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 1, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 1, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 1, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:45:47.264 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "26", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "0", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "selfmanaged", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 0, 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 0, 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [], 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:45:47.265 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:45:47.266 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "", 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:45:47.267 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 0, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {}, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "read_balance": { 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "score_type": "Fair distribution", 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "score_acting": 2.9900000095367432, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "score_stable": 2.9900000095367432, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_score": 0.67000001668930054, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_acting": 2, 2026-02-21T01:45:47.268 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_stable": 2, 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity_weighted": 1, 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity": 1, 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity_weighted": 1 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "831cb369-2a0c-4f59-9212-c85d454f3cab", 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:45:47.269 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 5, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 26, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 33, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 37, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 27, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:45:47.270 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.271 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:45:47.272 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:45:47.273 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1331655318 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.274 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/1331655318", 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/1331655318", 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1331655318", 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1331655318", 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "2936fbec-cae3-4d07-8448-09e5c05cd5b1", 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:45:47.275 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 9, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 20, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 28, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.276 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.277 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:45:47.278 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.279 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2672200289 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.280 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/2672200289", 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/2672200289", 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2672200289", 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2672200289", 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "c42ad328-be10-4fdf-8f0d-26b719fe5157", 2026-02-21T01:45:47.281 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 13, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 26, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 35, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 29, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.282 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.283 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:45:47.284 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.285 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1539761431 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.286 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/1539761431", 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/1539761431", 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1539761431", 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1539761431", 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:45:47.287 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:45:13.691714+0000", 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:44:39.061111+0000", 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:45:13.968946+0000", 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:45:47.288 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:45:14.254815+0000", 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:44:46.494731+0000", 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:45:47.289 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.290 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.291 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/1331655318,v1:127.0.0.1:6803/1331655318]", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:45:47.292 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:44:36.154081Z", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:45:47.293 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/1331655318,v1:127.0.0.1:6801/1331655318]", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1331655318,v1:127.0.0.1:6807/1331655318]", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1331655318,v1:127.0.0.1:6805/1331655318]", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:45:47.294 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/2672200289,v1:127.0.0.1:6813/2672200289]", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:45:47.295 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:44:39.907307Z", 2026-02-21T01:45:47.296 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/2672200289,v1:127.0.0.1:6811/2672200289]", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2672200289,v1:127.0.0.1:6817/2672200289]", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2672200289,v1:127.0.0.1:6815/2672200289]", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:45:47.297 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/1539761431,v1:127.0.0.1:6813/1539761431]", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:45:47.298 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:45:47.299 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:44:43.997851Z", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/1539761431,v1:127.0.0.1:6811/1539761431]", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1539761431,v1:127.0.0.1:6817/1539761431]", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1539761431,v1:127.0.0.1:6815/1539761431]", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:45:47.300 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 21, 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [ 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: "floor": 21 2026-02-21T01:45:47.301 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 39 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 38 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 40, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:45:47.302 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:45:47.303 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:45:47.304 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.305 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:45:47.306 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.307 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:45:47.308 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:45:47.309 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:45:47.310 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 0, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stderr:report 140873805 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:45:47.311 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:44:35:196236+0000", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:45:47.312 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 1, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51200, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:45:47.313 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 100, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 100, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:45:47.314 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 16, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 16, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "up": 2, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 2, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:45:47.315 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83852, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2072, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 27, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81700, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488948, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036682752, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2121728, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 421200, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 28654, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83660818 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 1, 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:45:47.316 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83852, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2072, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 27, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81700, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488948, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:45:47.317 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036682752, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2121728, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 421200, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 28654, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83660818 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 1, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.318 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51200, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 100, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 100, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:45:47.319 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1228800, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 153600, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 16, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 16, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "up": 2, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 2, 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.320 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 141733920776 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 38654705677 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 150323855368 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "num": 1 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:45:47.321 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 1, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 3, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:45:43.626568+0000", 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "services": { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "osd": { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "daemons": { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "summary": "", 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "0": { 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:45:47.322 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "2": { 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 36, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 220, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 306, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:45:47.323 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:45:47.324 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:45:47.324 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: expr 2 + 2 2026-02-21T01:45:47.324 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: objname=existing_4 2026-02-21T01:45:47.324 INFO:tasks.workunit.client.0.trial009.stdout:writing non-divergent object existing_4 2026-02-21T01:45:47.324 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:151: TEST_divergent: echo 'writing non-divergent object existing_4' 2026-02-21T01:45:47.324 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:152: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:47.476 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:47.476 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 26'100 40:136 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:47.477 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:47.477 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:47.477 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:47.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: dd if=/dev/urandom bs=1000 count=1 2026-02-21T01:45:47.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: rados -p test put existing_4 - 2026-02-21T01:45:47.486 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:45:47.486 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:45:47.486 INFO:tasks.workunit.client.0.trial009.stderr:1000 bytes (1.0 kB) copied, 1.7402e-05 s, 57.5 MB/s 2026-02-21T01:45:47.508 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery 2026-02-21T01:45:47.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:157: TEST_divergent: echo 'delay recovery' 2026-02-21T01:45:47.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:158: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:47.704 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:47.704 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 26'100 40:136 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:47.704 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:47.704 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:47.704 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: get_asok_path osd.0 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:47.713 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:47.714 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:45:47.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: CEPH_ARGS= 2026-02-21T01:45:47.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok set_recovery_delay 100000 2026-02-21T01:45:47.769 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: for i in $non_divergent 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: get_asok_path osd.2 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:47.776 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:47.777 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:45:47.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: CEPH_ARGS= 2026-02-21T01:45:47.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok set_recovery_delay 100000 2026-02-21T01:45:47.831 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:45:47.837 INFO:tasks.workunit.client.0.trial009.stdout:revive divergent 1 2026-02-21T01:45:47.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:165: TEST_divergent: echo 'revive divergent 1' 2026-02-21T01:45:47.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:166: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:48.043 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:48.043 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 26'100 40:136 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:48.043 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:48.044 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:48.044 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:48.052 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:167: TEST_divergent: ceph osd set noup 2026-02-21T01:45:48.362 INFO:tasks.workunit.client.0.trial009.stderr:noup is set 2026-02-21T01:45:48.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:168: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:45:48.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:48.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:45:48.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:45:48.380 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:45:48.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:45:48.395 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:48.390+0000 7f7b73f00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:48.397 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:48.394+0000 7f7b73f00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:48.398 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:48.394+0000 7f7b73f00a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:48.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-21T01:45:48.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:169: TEST_divergent: sleep 5 2026-02-21T01:45:48.794 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:48.790+0000 7f7b73f00a40 -1 Falling back to public interface 2026-02-21T01:45:49.418 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:49.414+0000 7f7b73f00a40 -1 osd.1 26 log_to_monitors true 2026-02-21T01:45:53.597 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery divergent 2026-02-21T01:45:53.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:171: TEST_divergent: echo 'delay recovery divergent' 2026-02-21T01:45:53.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:172: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:53.794 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:53.794 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 40'101 43:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:53.794 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:53.794 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:53.795 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:53.803 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: get_asok_path osd.1 2026-02-21T01:45:53.803 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:45:53.803 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:45:53.803 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:45:53.803 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:53.804 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:53.804 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:45:53.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: CEPH_ARGS= 2026-02-21T01:45:53.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok set_recovery_delay 100000 2026-02-21T01:45:53.859 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:45:53.865 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: ceph osd unset noup 2026-02-21T01:45:54.187 INFO:tasks.workunit.client.0.trial009.stderr:noup is unset 2026-02-21T01:45:54.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:177: TEST_divergent: wait_for_osd up 0 2026-02-21T01:45:54.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:45:54.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:45:54.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:45:54.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:45:54.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:54.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:45:54.200 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:45:54.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:54.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:45:54.424 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 33 up_thru 37 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6800/1331655318,v1:127.0.0.1:6801/1331655318] [v2:127.0.0.1:6802/1331655318,v1:127.0.0.1:6803/1331655318] exists,up 831cb369-2a0c-4f59-9212-c85d454f3cab 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:178: TEST_divergent: wait_for_osd up 1 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:54.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:45:54.427 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:45:54.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:54.427 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:45:54.635 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/4078073546,v1:127.0.0.1:6819/4078073546] [v2:127.0.0.1:6820/4078073546,v1:127.0.0.1:6821/4078073546] exists,up 2936fbec-cae3-4d07-8448-09e5c05cd5b1 2026-02-21T01:45:54.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:45:54.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:45:54.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:45:54.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:179: TEST_divergent: wait_for_osd up 2 2026-02-21T01:45:54.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:45:54.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:45:54.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:45:54.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:45:54.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:54.639 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:45:54.640 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:45:54.640 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:54.640 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:45:54.839 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 35 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/1539761431,v1:127.0.0.1:6811/1539761431] [v2:127.0.0.1:6812/1539761431,v1:127.0.0.1:6813/1539761431] exists,up c42ad328-be10-4fdf-8f0d-26b719fe5157 2026-02-21T01:45:54.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:45:54.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:45:54.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:45:54.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:181: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:55.034 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:55.034 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 40'101 43:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:55.034 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:55.034 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:55.034 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:55.042 INFO:tasks.workunit.client.0.trial009.stdout:wait for peering 2026-02-21T01:45:55.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:182: TEST_divergent: echo 'wait for peering' 2026-02-21T01:45:55.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:183: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:55.260 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:55.260 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 40'101 43:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:55.260 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:55.261 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:55.261 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:55.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:184: TEST_divergent: rados -p test put foo /tmp/tmp.ussY4xfZwU 2026-02-21T01:45:55.293 INFO:tasks.workunit.client.0.trial009.stdout:killing divergent 1 2026-02-21T01:45:55.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:186: TEST_divergent: echo 'killing divergent 1' 2026-02-21T01:45:55.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:187: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:55.499 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:55.499 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:45:19.011822+0000 40'101 43:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:10.770080+0000 0 0 2026-02-21T01:45:55.499 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:55.500 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:55.500 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:55.508 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:188: TEST_divergent: kill_daemons td/divergent-priors KILL osd.1 2026-02-21T01:45:55.508 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:45:55.508 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:45:55.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:45:55.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:45:55.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:45:55.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:45:55.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:190: TEST_divergent: echo 'reviving divergent 1' 2026-02-21T01:45:55.617 INFO:tasks.workunit.client.0.trial009.stdout:reviving divergent 1 2026-02-21T01:45:55.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:191: TEST_divergent: ceph pg dump pgs 2026-02-21T01:45:55.813 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:45:55.813 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 2 2 0 0 51688 0 0 17 84 17 active+recovering+degraded 2026-02-21T01:45:54.291864+0000 40'101 46:144 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:45:55.813 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:45:55.813 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:45:55.813 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:192: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:45:55.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:45:55.825 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:45:55.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:45:55.827 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:45:55.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:45:55.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:45:55.827 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:45:55.828 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:45:55.828 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:45:55.829 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:45:55.829 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:45:55.851 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:55.846+0000 7fc64a0e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:55.852 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:55.850+0000 7fc64a0e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:55.853 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:55.850+0000 7fc64a0e4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:45:56.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:45:56.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:45:56.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:45:56.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:45:56.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:45:56.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:45:56.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:45:56.038 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:45:56.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:45:56.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:45:56.221 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:56.218+0000 7fc64a0e4a40 -1 Falling back to public interface 2026-02-21T01:45:56.256 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/4078073546,v1:127.0.0.1:6819/4078073546] [v2:127.0.0.1:6820/4078073546,v1:127.0.0.1:6821/4078073546] exists,up 2936fbec-cae3-4d07-8448-09e5c05cd5b1 2026-02-21T01:45:56.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:45:56.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:45:56.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:45:56.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:194: TEST_divergent: sleep 20 2026-02-21T01:45:56.840 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:45:56.838+0000 7fc64a0e4a40 -1 osd.1 47 log_to_monitors true 2026-02-21T01:46:16.258 INFO:tasks.workunit.client.0.trial009.stdout:allowing recovery 2026-02-21T01:46:16.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:196: TEST_divergent: echo 'allowing recovery' 2026-02-21T01:46:16.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:197: TEST_divergent: ceph pg dump pgs 2026-02-21T01:46:16.464 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:46:16.464 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 52200 0 0 18 84 18 active+clean 2026-02-21T01:45:57.347469+0000 47'102 51:163 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T13:20:53.573893+0000 0 0 2026-02-21T01:46:16.465 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:46:16.465 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:46:16.465 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:46:16.473 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: for i in $osds 2026-02-21T01:46:16.473 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-21T01:46:16.564 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:46:16.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: for i in $osds 2026-02-21T01:46:16.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-21T01:46:16.644 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:46:16.651 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: for i in $osds 2026-02-21T01:46:16.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-21T01:46:16.737 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:46:16.745 INFO:tasks.workunit.client.0.trial009.stdout:reading divergent objects 2026-02-21T01:46:16.745 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:204: TEST_divergent: echo 'reading divergent objects' 2026-02-21T01:46:16.745 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:205: TEST_divergent: ceph pg dump pgs 2026-02-21T01:46:16.954 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:46:16.954 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 52200 0 0 18 84 18 active+clean 2026-02-21T01:45:57.347469+0000 47'102 51:163 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:44:48.353076+0000 0'0 2026-02-21T01:44:48.353076+0000 0 0 periodic scrub scheduled @ 2026-02-22T13:20:53.573893+0000 0 0 2026-02-21T01:46:16.954 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:46:16.954 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:46:16.954 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:46:16.963 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: expr 2 + 2 2026-02-21T01:46:16.964 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: seq 1 4 2026-02-21T01:46:16.965 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:46:16.965 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_1 td/divergent-priors/existing 2026-02-21T01:46:16.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:46:16.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_2 td/divergent-priors/existing 2026-02-21T01:46:17.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:46:17.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_3 td/divergent-priors/existing 2026-02-21T01:46:17.042 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:46:17.042 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: rados -p test get existing_4 td/divergent-priors/existing 2026-02-21T01:46:17.070 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rm -f td/divergent-priors/existing 2026-02-21T01:46:17.071 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-21T01:46:17.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4534.0:1 2026-02-21T01:45:02.569076+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 20 _merge_object_divergent_entries: keeping 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4534.0:1 2026-02-21T01:45:02.569076+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 26'100 first_divergent_update: 26'101 last_divergent_update: 26'101 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 26'101 2026-02-21T01:46:17.077 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 26'101 (26'100) modify 1:c156033e:::existing_100:head by client.4534.0:1 2026-02-21T01:45:02.569076+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-21T01:46:17.078 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:45:54.258+0000 7f7b55685640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-21T01:46:17.078 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-21T01:46:17.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: grep -q '_merge_object_divergent_entries.*cannot roll back, removing and adding to missing' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:46:17.081 INFO:tasks.workunit.client.0.trial009.stdout:success 2026-02-21T01:46:17.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:219: TEST_divergent: echo success 2026-02-21T01:46:17.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:221: TEST_divergent: rm -f /tmp/tmp.ussY4xfZwU 2026-02-21T01:46:17.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:222: TEST_divergent: delete_pool test 2026-02-21T01:46:17.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-21T01:46:17.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-21T01:46:17.342 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' does not exist 2026-02-21T01:46:17.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:223: TEST_divergent: kill_daemons td/divergent-priors 2026-02-21T01:46:17.352 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:46:17.352 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:46:17.352 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:46:17.352 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:46:17.352 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:46:22.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:46:22.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-21T01:46:22.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:46:22.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:46:22.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:46:22.670 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:46:22.670 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:46:22.670 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:46:22.670 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:46:22.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:46:22.676 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:46:22.677 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:46:22.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:46:22.678 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:46:22.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:46:22.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:46:22.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:46:22.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:46:22.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:46:22.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:46:22.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:46:22.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:46:22.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:46:22.684 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:46:22.702 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:46:22.702 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:22.702 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:22.702 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:46:22.705 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:46:22.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:46:22.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:46:22.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:46:22.707 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:46:22.707 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:46:22.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:46:22.708 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:46:22.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:46:22.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:46:22.710 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:46:22.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:46:22.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:46:22.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:46:22.712 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:46:22.713 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:46:22.713 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:46:22.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:46:22.715 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:46:22.715 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:22.715 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:22.715 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:46:22.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:46:22.717 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:46:22.717 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-21T01:46:22.718 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:46:22.718 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:22.718 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:22.718 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.20522 2026-02-21T01:46:22.720 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:46:22.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T01:46:22.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:46:22.721 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-21T01:46:22.721 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_2 td/divergent-priors 2026-02-21T01:46:22.721 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:422: TEST_divergent_2: local dir=td/divergent-priors 2026-02-21T01:46:22.721 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: file_with_random_data 2026-02-21T01:46:22.721 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=512 2026-02-21T01:46:22.722 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:46:22.722 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:22.722 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.T4YSTHp4YX bs=512 count=1 2026-02-21T01:46:22.723 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:46:22.724 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:46:22.724 INFO:tasks.workunit.client.0.trial009.stderr:512 bytes copied, 8.8224e-05 s, 5.8 MB/s 2026-02-21T01:46:22.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:22.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: local dummyfile=/tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:22.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:426: TEST_divergent_2: file_with_random_data 1000 2026-02-21T01:46:22.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=1000 2026-02-21T01:46:22.725 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:46:22.726 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.ldUC5EQFKa 2026-02-21T01:46:22.726 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.ldUC5EQFKa bs=1000 count=1 2026-02-21T01:46:22.727 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:46:22.727 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:46:22.727 INFO:tasks.workunit.client.0.trial009.stderr:1000 bytes (1.0 kB) copied, 8.2122e-05 s, 12.2 MB/s 2026-02-21T01:46:22.727 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.ldUC5EQFKa 2026-02-21T01:46:22.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:426: TEST_divergent_2: local dummyfile2=/tmp/tmp.ldUC5EQFKa 2026-02-21T01:46:22.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: local num_osds=3 2026-02-21T01:46:22.728 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: expr 3 - 1 2026-02-21T01:46:22.729 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: seq 0 2 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: local 'osds=0 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:430: TEST_divergent_2: run_mon td/divergent-priors a 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:46:22.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:46:22.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:46:22.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-21T01:46:22.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-21T01:46:22.753 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:46:22.754 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:46:22.754 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:46:22.754 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:46:22.754 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:22.754 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:22.755 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:22.755 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:46:22.782 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:46:22.783 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:46:22.784 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:46:22.784 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:46:22.784 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:46:22.784 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:46:22.785 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:46:22.785 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:46:22.785 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:46:22.785 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:46:22.786 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:22.786 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:22.786 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:46:22.786 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:46:22.786 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get fsid 2026-02-21T01:46:22.840 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:46:22.840 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:46:22.840 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:46:22.840 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:46:22.840 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:46:22.841 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:46:22.841 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:46:22.841 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:46:22.841 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:46:22.842 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:22.842 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:22.842 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:46:22.842 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:46:22.842 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get mon_host 2026-02-21T01:46:22.895 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: run_mgr td/divergent-priors x 2026-02-21T01:46:22.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-21T01:46:22.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:46:22.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:46:22.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:46:22.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-21T01:46:22.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:46:23.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:46:23.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:46:23.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:46:23.007 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:46:23.007 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:23.007 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:23.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:23.008 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:46:23.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:46:23.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:432: TEST_divergent_2: for i in $osds 2026-02-21T01:46:23.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:434: TEST_divergent_2: run_osd td/divergent-priors 0 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:46:23.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:46:23.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:46:23.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:46:23.037 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:46:23.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6f27f87b-8ce5-4bda-b3d5-0b765636ec02 2026-02-21T01:46:23.037 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 6f27f87b-8ce5-4bda-b3d5-0b765636ec02 2026-02-21T01:46:23.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6f27f87b-8ce5-4bda-b3d5-0b765636ec02' 2026-02-21T01:46:23.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:46:23.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBvDplpZnG+AhAAQckBoQeU/pqZPuadycWr1A== 2026-02-21T01:46:23.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBvDplpZnG+AhAAQckBoQeU/pqZPuadycWr1A=="}' 2026-02-21T01:46:23.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6f27f87b-8ce5-4bda-b3d5-0b765636ec02 -i td/divergent-priors/0/new.json 2026-02-21T01:46:23.164 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:46:23.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-21T01:46:23.173 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBvDplpZnG+AhAAQckBoQeU/pqZPuadycWr1A== --osd-uuid 6f27f87b-8ce5-4bda-b3d5-0b765636ec02 2026-02-21T01:46:23.193 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:23.189+0000 7f23b80faa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:23.195 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:23.189+0000 7f23b80faa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:23.196 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:23.189+0000 7f23b80faa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:23.196 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:23.193+0000 7f23b80faa40 -1 bdev(0x55b718f68c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:46:23.196 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:23.193+0000 7f23b80faa40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-21T01:46:23.928 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-21T01:46:23.928 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:46:23.929 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:46:23.929 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:46:23.930 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:46:24.052 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:46:24.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:46:24.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:46:24.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:46:24.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:46:24.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:46:24.071 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:24.065+0000 7fa228251a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:24.072 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:24.069+0000 7fa228251a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:24.074 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:24.069+0000 7fa228251a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:46:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:46:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:46:24.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:46:24.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:46:24.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:24.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:46:24.164 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:46:24.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:24.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:46:24.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:46:24.521 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:24.517+0000 7fa228251a40 -1 Falling back to public interface 2026-02-21T01:46:25.238 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:25.233+0000 7fa228251a40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:46:25.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:46:25.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:25.273 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:46:25.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:46:25.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:25.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:46:25.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:46:26.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:46:26.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:26.383 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:46:26.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:46:26.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:26.384 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:46:26.590 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/10251172,v1:127.0.0.1:6801/10251172] [v2:127.0.0.1:6802/10251172,v1:127.0.0.1:6803/10251172] exists,up 6f27f87b-8ce5-4bda-b3d5-0b765636ec02 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:432: TEST_divergent_2: for i in $osds 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:434: TEST_divergent_2: run_osd td/divergent-priors 1 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:46:26.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:46:26.595 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:46:26.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:46:26.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:46:26.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:46:26.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:46:26.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:46:26.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:46:26.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:46:26.598 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:46:26.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 13b34d7e-7523-41ba-bf17-153ca2241744' 2026-02-21T01:46:26.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:46:26.609 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQByDplpFaM4JBAAwfeEAKbwnN66WFzYKF1xGQ== 2026-02-21T01:46:26.610 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQByDplpFaM4JBAAwfeEAKbwnN66WFzYKF1xGQ=="}' 2026-02-21T01:46:26.610 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 13b34d7e-7523-41ba-bf17-153ca2241744 -i td/divergent-priors/1/new.json 2026-02-21T01:46:26.829 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:46:26.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-21T01:46:26.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQByDplpFaM4JBAAwfeEAKbwnN66WFzYKF1xGQ== --osd-uuid 13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:46:26.869 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:26.865+0000 7fd0b9487a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:26.870 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:26.865+0000 7fd0b9487a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:26.871 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:26.865+0000 7fd0b9487a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:26.872 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:26.865+0000 7fd0b9487a40 -1 bdev(0x55db2a34ec00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:46:26.872 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:26.865+0000 7fd0b9487a40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-21T01:46:27.543 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-21T01:46:27.543 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:46:27.543 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:46:27.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:46:27.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:46:27.831 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:46:27.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:46:27.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:46:27.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:46:27.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:46:27.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:46:27.850 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:27.845+0000 7f4a6a8c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:27.851 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:27.845+0000 7f4a6a8c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:27.852 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:27.849+0000 7f4a6a8c8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:28.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:46:28.050 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:46:28.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:28.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:46:28.209 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:28.205+0000 7f4a6a8c8a40 -1 Falling back to public interface 2026-02-21T01:46:28.254 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:46:28.814 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:28.809+0000 7f4a6a8c8a40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:46:29.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:46:29.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:29.256 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:46:29.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:46:29.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:29.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:46:29.463 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/134190718,v1:127.0.0.1:6811/134190718] [v2:127.0.0.1:6812/134190718,v1:127.0.0.1:6813/134190718] exists,up 13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:46:29.467 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:46:29.467 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:46:29.467 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:46:29.467 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:432: TEST_divergent_2: for i in $osds 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:434: TEST_divergent_2: run_osd td/divergent-priors 2 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:46:29.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:29.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:46:29.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:46:29.471 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:46:29.471 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 2026-02-21T01:46:29.471 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 2026-02-21T01:46:29.472 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74' 2026-02-21T01:46:29.472 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:46:29.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB1DplpiQekHBAAyRMXVcVNUWVnCX6FhD9KgA== 2026-02-21T01:46:29.483 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB1DplpiQekHBAAyRMXVcVNUWVnCX6FhD9KgA=="}' 2026-02-21T01:46:29.483 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 -i td/divergent-priors/2/new.json 2026-02-21T01:46:29.699 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:46:29.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-21T01:46:29.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB1DplpiQekHBAAyRMXVcVNUWVnCX6FhD9KgA== --osd-uuid d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 2026-02-21T01:46:29.728 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:29.725+0000 7ff3b511da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:29.730 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:29.725+0000 7ff3b511da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:29.731 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:29.725+0000 7ff3b511da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:29.732 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:29.725+0000 7ff3b511da40 -1 bdev(0x559928da2c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:46:29.732 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:29.729+0000 7ff3b511da40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-21T01:46:30.410 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-21T01:46:30.411 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:46:30.412 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:46:30.412 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:46:30.412 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:46:30.692 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:46:30.693 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:46:30.693 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:46:30.693 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:46:30.693 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:46:30.693 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:46:30.712 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:30.709+0000 7fc8cc74fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:30.713 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:30.709+0000 7fc8cc74fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:30.714 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:30.709+0000 7fc8cc74fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:46:30.911 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:46:30.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:46:30.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:46:30.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:46:30.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:46:30.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:30.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:46:30.913 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:46:30.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:30.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:46:31.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:46:31.145 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:31.141+0000 7fc8cc74fa40 -1 Falling back to public interface 2026-02-21T01:46:31.753 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:46:31.749+0000 7fc8cc74fa40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:46:32.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:46:32.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:32.133 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:46:32.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:46:32.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:32.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:46:32.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:46:33.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:46:33.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:46:33.345 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:46:33.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:46:33.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:46:33.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:46:33.566 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/568847785,v1:127.0.0.1:6819/568847785] [v2:127.0.0.1:6820/568847785,v1:127.0.0.1:6821/568847785] exists,up d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 2026-02-21T01:46:33.567 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:46:33.567 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:46:33.567 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:46:33.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:437: TEST_divergent_2: ceph osd set noout 2026-02-21T01:46:33.858 INFO:tasks.workunit.client.0.trial009.stderr:noout is set 2026-02-21T01:46:33.870 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:438: TEST_divergent_2: ceph osd set noin 2026-02-21T01:46:34.162 INFO:tasks.workunit.client.0.trial009.stderr:noin is set 2026-02-21T01:46:34.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:439: TEST_divergent_2: ceph osd set nodown 2026-02-21T01:46:34.467 INFO:tasks.workunit.client.0.trial009.stderr:nodown is set 2026-02-21T01:46:34.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:440: TEST_divergent_2: create_pool test 1 1 2026-02-21T01:46:34.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-21T01:46:34.721 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' already exists 2026-02-21T01:46:34.733 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T01:46:35.734 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:441: TEST_divergent_2: ceph osd pool set test size 3 2026-02-21T01:46:36.046 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 size to 3 2026-02-21T01:46:36.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:442: TEST_divergent_2: ceph osd pool set test min_size 2 2026-02-21T01:46:36.350 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 min_size to 2 2026-02-21T01:46:36.361 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:444: TEST_divergent_2: flush_pg_stats 2026-02-21T01:46:36.361 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:46:36.362 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:46:36.574 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:46:36.574 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:46:36.575 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:46:36.575 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:46:36.575 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:36.575 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:46:36.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-21T01:46:36.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-21T01:46:36.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-21T01:46:36.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:36.657 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:46:36.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-21T01:46:36.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-21T01:46:36.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-21T01:46:36.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:36.736 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:46:36.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542147 2026-02-21T01:46:36.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542147 2026-02-21T01:46:36.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-60129542147' 2026-02-21T01:46:36.817 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:36.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-21T01:46:36.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:36.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:46:36.819 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-21T01:46:36.819 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:36.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-21T01:46:36.820 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836485 2026-02-21T01:46:36.821 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-21T01:46:36.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:46:37.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836485 2026-02-21T01:46:37.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:46:38.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:46:38.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:46:38.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-21T01:46:38.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:38.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-21T01:46:38.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:38.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:46:38.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-21T01:46:38.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:38.261 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-21T01:46:38.262 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-21T01:46:38.262 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705668 2026-02-21T01:46:38.262 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:46:38.479 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-21T01:46:38.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:38.480 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542147 2026-02-21T01:46:38.480 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:38.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:46:38.482 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542147 2026-02-21T01:46:38.482 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:38.483 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542147 2026-02-21T01:46:38.483 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-21T01:46:38.484 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 60129542147 2026-02-21T01:46:38.484 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:46:38.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-21T01:46:38.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:445: TEST_divergent_2: wait_for_clean 2026-02-21T01:46:38.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:46:38.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:46:38.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:46:38.703 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:46:38.704 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:46:38.704 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:46:38.705 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:46:38.705 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:46:38.705 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:46:38.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:46:38.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:46:38.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:46:38.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:46:38.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:46:38.804 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:46:39.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:46:39.027 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:46:39.027 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:46:39.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:46:39.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:39.028 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:46:39.108 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-21T01:46:39.108 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-21T01:46:39.108 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-21T01:46:39.108 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:39.109 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:46:39.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705670 2026-02-21T01:46:39.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705670 2026-02-21T01:46:39.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705670' 2026-02-21T01:46:39.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:39.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:46:39.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542150 2026-02-21T01:46:39.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542150 2026-02-21T01:46:39.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-60129542150' 2026-02-21T01:46:39.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:39.275 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-21T01:46:39.275 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:39.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:46:39.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-21T01:46:39.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:39.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-21T01:46:39.278 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836487 2026-02-21T01:46:39.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-21T01:46:39.279 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:46:39.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-21T01:46:39.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:39.498 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705670 2026-02-21T01:46:39.498 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:39.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:46:39.500 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705670 2026-02-21T01:46:39.500 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:39.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705670 2026-02-21T01:46:39.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-21T01:46:39.502 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705670 2026-02-21T01:46:39.502 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:46:39.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-21T01:46:39.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:39.727 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542150 2026-02-21T01:46:39.728 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:39.729 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:46:39.729 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542150 2026-02-21T01:46:39.729 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:39.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542150 2026-02-21T01:46:39.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-21T01:46:39.731 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 60129542150 2026-02-21T01:46:39.731 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:46:39.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542150 2026-02-21T01:46:39.947 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:46:40.947 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:46:40.948 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:46:41.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542148 -lt 60129542150 2026-02-21T01:46:41.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:46:42.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T01:46:42.169 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:46:42.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-21T01:46:42.389 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:46:42.389 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:46:42.389 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:46:42.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:46:42.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:46:42.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:46:42.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:46:42.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:46:42.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:46:42.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:46:42.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:46:42.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:46:42.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:46:42.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:46:42.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:46:43.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:46:43.180 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:46:43.180 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:46:43.180 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: ceph pg dump pgs --format=json 2026-02-21T01:46:43.180 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: jq '.pg_stats[0].up_primary' 2026-02-21T01:46:43.375 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:46:43.386 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: local divergent=1 2026-02-21T01:46:43.386 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:449: TEST_divergent_2: echo 'primary and soon to be divergent is 1' 2026-02-21T01:46:43.386 INFO:tasks.workunit.client.0.trial009.stdout:primary and soon to be divergent is 1 2026-02-21T01:46:43.387 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:450: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:46:43.589 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:46:43.589 INFO:tasks.workunit.client.0.trial009.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T01:46:35.216591+0000 0'0 28:22 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:17:13.142135+0000 0 0 2026-02-21T01:46:43.590 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:46:43.590 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:46:43.590 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:46:43.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: local non_divergent= 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:452: TEST_divergent_2: for i in $osds 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: '[' 0 = 1 ']' 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:457: TEST_divergent_2: non_divergent=' 0' 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:452: TEST_divergent_2: for i in $osds 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: '[' 1 = 1 ']' 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:455: TEST_divergent_2: continue 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:452: TEST_divergent_2: for i in $osds 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: '[' 2 = 1 ']' 2026-02-21T01:46:43.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:457: TEST_divergent_2: non_divergent=' 0 2' 2026-02-21T01:46:43.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:460: TEST_divergent_2: echo 'writing initial objects' 2026-02-21T01:46:43.600 INFO:tasks.workunit.client.0.trial009.stdout:writing initial objects 2026-02-21T01:46:43.600 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: seq 1 100 2026-02-21T01:46:43.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_1 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.622 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.622 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_2 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_3 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_4 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_5 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.730 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.730 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_6 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_7 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_8 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_9 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_10 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.858 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.858 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_11 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.886 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.886 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_12 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_13 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_14 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_15 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:43.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:43.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_16 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_17 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_18 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_19 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_20 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_21 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_22 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_23 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.196 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.196 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_24 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_25 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_26 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_27 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.299 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.299 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_28 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.326 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.326 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_29 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_30 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.384 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.384 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_31 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_32 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.432 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.432 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_33 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_34 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.484 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_35 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_36 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_37 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_38 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_39 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.625 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_40 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_41 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_42 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_43 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_44 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_45 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.792 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.792 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_46 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_47 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_48 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_49 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.904 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.904 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_50 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.932 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_51 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_52 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:44.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:44.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_53 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_54 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_55 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_56 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_57 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.105 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.105 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_58 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_59 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_60 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_61 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_62 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.245 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.245 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_63 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_64 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_65 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_66 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_67 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_68 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_69 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_70 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_71 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_72 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.506 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.507 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_73 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_74 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_75 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_76 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_77 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_78 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.661 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.661 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_79 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.688 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.689 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_80 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_81 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_82 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_83 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_84 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_85 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_86 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.857 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.857 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_87 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.885 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_88 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_89 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_90 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.968 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.968 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_91 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:45.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:45.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_92 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_93 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_94 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_95 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.105 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.105 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_96 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_97 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_98 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.189 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_99 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.209 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:462: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-21T01:46:46.209 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:464: TEST_divergent_2: rados -p test put existing_100 /tmp/tmp.T4YSTHp4YX 2026-02-21T01:46:46.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:467: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:46:46.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:467: TEST_divergent_2: wait_for_clean 2026-02-21T01:46:46.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:46:46.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:46:46.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:46:46.238 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:46:46.239 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:46:46.239 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:46:46.239 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:46:46.240 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:46:46.240 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:46:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:46:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:46:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:46:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:46:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:46:46.311 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:46:46.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:46:46.529 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:46:46.529 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:46:46.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:46:46.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:46.529 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:46:46.606 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-21T01:46:46.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-21T01:46:46.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-21T01:46:46.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:46.607 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:46:46.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705674 2026-02-21T01:46:46.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705674 2026-02-21T01:46:46.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674' 2026-02-21T01:46:46.688 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:46:46.688 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:46:46.769 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=60129542153 2026-02-21T01:46:46.769 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 60129542153 2026-02-21T01:46:46.769 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-60129542153' 2026-02-21T01:46:46.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:46.770 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-21T01:46:46.770 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:46.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:46:46.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-21T01:46:46.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:46.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-21T01:46:46.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-21T01:46:46.774 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836491 2026-02-21T01:46:46.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:46:46.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836491 2026-02-21T01:46:46.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:46:48.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:46:48.000 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:46:48.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-21T01:46:48.218 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:48.218 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705674 2026-02-21T01:46:48.219 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:48.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:46:48.220 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705674 2026-02-21T01:46:48.220 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:48.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705674 2026-02-21T01:46:48.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-21T01:46:48.222 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705674 2026-02-21T01:46:48.222 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:46:48.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-21T01:46:48.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:46:48.453 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-60129542153 2026-02-21T01:46:48.453 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:46:48.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:46:48.454 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-60129542153 2026-02-21T01:46:48.455 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:46:48.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=60129542153 2026-02-21T01:46:48.456 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 60129542153 2026-02-21T01:46:48.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-21T01:46:48.457 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:46:48.674 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-21T01:46:48.675 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:46:48.675 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:46:48.675 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:46:48.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:46:48.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:46:48.950 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:46:48.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:46:48.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:46:48.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:46:48.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:46:48.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:46:49.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:46:49.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:46:49.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:46:49.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:46:49.437 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:46:49.437 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:46:49.437 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:46:49.437 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:469: TEST_divergent_2: get_pg test existing_1 2026-02-21T01:46:49.437 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-21T01:46:49.438 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-21T01:46:49.438 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-21T01:46:49.438 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-21T01:46:49.650 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:469: TEST_divergent_2: local pgid=1.0 2026-02-21T01:46:49.650 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:472: TEST_divergent_2: echo 'blackholing osds 0 2' 2026-02-21T01:46:49.651 INFO:tasks.workunit.client.0.trial009.stdout:blackholing osds 0 2 2026-02-21T01:46:49.651 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:46:49.851 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:46:49.851 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-21T01:46:35.216591+0000 28'100 28:122 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:17:13.142135+0000 0 0 2026-02-21T01:46:49.851 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:46:49.852 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:46:49.852 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:46:49.860 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:474: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:46:49.861 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: get_asok_path osd.0 2026-02-21T01:46:49.861 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:46:49.861 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:46:49.861 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:46:49.861 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:49.861 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:49.862 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:46:49.862 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: CEPH_ARGS= 2026-02-21T01:46:49.862 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-21T01:46:49.914 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:46:49.915 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:46:49.915 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:46:49.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:474: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:46:49.921 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: get_asok_path osd.2 2026-02-21T01:46:49.921 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:46:49.921 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:46:49.922 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:46:49.922 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:46:49.922 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:46:49.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:46:49.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: CEPH_ARGS= 2026-02-21T01:46:49.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:476: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-21T01:46:49.974 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:46:49.974 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:46:49.974 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:46:49.979 INFO:tasks.workunit.client.0.trial009.stdout:create new divergent objects 2026-02-21T01:46:49.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: echo 'create new divergent objects' 2026-02-21T01:46:49.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:481: TEST_divergent_2: seq 1 2 2026-02-21T01:46:49.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:481: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:46:49.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:481: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:46:49.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:483: TEST_divergent_2: rados -p test create newobject_1 2026-02-21T01:46:49.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: echo 'writing divergent objects' 2026-02-21T01:46:49.982 INFO:tasks.workunit.client.0.trial009.stdout:writing divergent objects 2026-02-21T01:46:49.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:483: TEST_divergent_2: rados -p test create newobject_2 2026-02-21T01:46:49.982 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:487: TEST_divergent_2: seq 1 2 2026-02-21T01:46:49.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:487: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-21T01:46:49.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:487: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-21T01:46:49.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:489: TEST_divergent_2: rados -p test put existing_1 /tmp/tmp.ldUC5EQFKa 2026-02-21T01:46:49.984 INFO:tasks.workunit.client.0.trial009.stdout:remove divergent objects 2026-02-21T01:46:49.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: echo 'remove divergent objects' 2026-02-21T01:46:49.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:489: TEST_divergent_2: rados -p test put existing_2 /tmp/tmp.ldUC5EQFKa 2026-02-21T01:46:49.985 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:493: TEST_divergent_2: seq 1 2 2026-02-21T01:46:49.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:493: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-21T01:46:49.985 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: expr 1 + 2 2026-02-21T01:46:49.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rmi=3 2026-02-21T01:46:49.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:493: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-21T01:46:49.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:496: TEST_divergent_2: rados -p test rm existing_3 2026-02-21T01:46:49.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: expr 2 + 2 2026-02-21T01:46:49.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rmi=4 2026-02-21T01:46:49.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:498: TEST_divergent_2: sleep 10 2026-02-21T01:46:49.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:496: TEST_divergent_2: rados -p test rm existing_4 2026-02-21T01:46:59.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:499: TEST_divergent_2: killall -9 rados 2026-02-21T01:47:00.040 INFO:tasks.workunit.client.0.trial009.stderr:rados: no process found 2026-02-21T01:47:00.041 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:502: TEST_divergent_2: echo 'killing all the osds' 2026-02-21T01:47:00.041 INFO:tasks.workunit.client.0.trial009.stdout:killing all the osds 2026-02-21T01:47:00.041 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:503: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:47:00.247 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:47:00.248 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 active+clean 2026-02-21T01:46:35.216591+0000 28'106 28:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:17:13.142135+0000 0 0 2026-02-21T01:47:00.248 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:47:00.248 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:47:00.248 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:47:00.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd 2026-02-21T01:47:00.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:47:00.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:47:00.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:47:00.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:47:00.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:47:00.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:47:00.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:505: TEST_divergent_2: for i in $osds 2026-02-21T01:47:00.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:507: TEST_divergent_2: ceph osd down osd.0 2026-02-21T01:47:00.827 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:47:00.839 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:505: TEST_divergent_2: for i in $osds 2026-02-21T01:47:00.839 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:507: TEST_divergent_2: ceph osd down osd.1 2026-02-21T01:47:01.101 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:47:01.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:505: TEST_divergent_2: for i in $osds 2026-02-21T01:47:01.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:507: TEST_divergent_2: ceph osd down osd.2 2026-02-21T01:47:01.374 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:47:01.385 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:509: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:47:01.386 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:511: TEST_divergent_2: ceph osd out osd.0 2026-02-21T01:47:01.652 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already out. 2026-02-21T01:47:01.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:509: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:47:01.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:511: TEST_divergent_2: ceph osd out osd.2 2026-02-21T01:47:01.928 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already out. 2026-02-21T01:47:01.939 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:515: TEST_divergent_2: echo 'bringing up non_divergent 0 2' 2026-02-21T01:47:01.940 INFO:tasks.workunit.client.0.trial009.stdout:bringing up non_divergent 0 2 2026-02-21T01:47:01.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:47:02.140 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:47:02.140 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 stale+active+clean 2026-02-21T01:46:35.216591+0000 28'106 28:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T02:17:13.142135+0000 0 0 2026-02-21T01:47:02.140 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:47:02.140 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:47:02.140 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:517: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:519: TEST_divergent_2: activate_osd td/divergent-priors 0 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:47:02.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:47:02.152 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:47:02.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:47:02.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:47:02.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:47:02.154 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:47:02.155 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:47:02.155 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:47:02.155 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-21T01:47:02.155 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:47:02.156 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:47:02.156 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:47:02.156 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:47:02.179 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:02.175+0000 7f64e16dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:47:02.180 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:02.175+0000 7f64e16dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:47:02.181 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:02.175+0000 7f64e16dfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:47:02.370 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:47:02.371 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:47:02.371 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:47:02.371 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:47:02.371 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:47:02.371 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:47:02.371 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:47:02.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:47:02.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:47:02.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:47:02.579 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:02.575+0000 7f64e16dfa40 -1 Falling back to public interface 2026-02-21T01:47:02.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:47:03.288 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:03.283+0000 7f64e16dfa40 -1 osd.0 28 log_to_monitors true 2026-02-21T01:47:03.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:47:03.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:47:03.588 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:47:03.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:47:03.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:47:03.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:47:03.794 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6800/3496815257,v1:127.0.0.1:6801/3496815257] [v2:127.0.0.1:6802/3496815257,v1:127.0.0.1:6803/3496815257] exists,up 6f27f87b-8ce5-4bda-b3d5-0b765636ec02 2026-02-21T01:47:03.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:517: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:519: TEST_divergent_2: activate_osd td/divergent-priors 2 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:47:03.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:47:03.799 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:47:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:47:03.801 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:47:03.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:47:03.825 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:03.819+0000 7fbd5cc6da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:47:03.826 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:03.823+0000 7fbd5cc6da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:47:03.827 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:03.823+0000 7fbd5cc6da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:47:04.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:47:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:47:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:47:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:47:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:47:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:47:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:47:04.021 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:47:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:47:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:47:04.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:47:04.267 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:04.263+0000 7fbd5cc6da40 -1 Falling back to public interface 2026-02-21T01:47:04.919 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:47:04.915+0000 7fbd5cc6da40 -1 osd.2 28 log_to_monitors true 2026-02-21T01:47:05.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:47:05.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:47:05.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:47:05.234 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:47:05.234 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:47:05.234 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:47:05.449 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up out weight 0 up_from 38 up_thru 0 down_at 31 last_clean_interval [14,28) [v2:127.0.0.1:6810/2328339827,v1:127.0.0.1:6811/2328339827] [v2:127.0.0.1:6812/2328339827,v1:127.0.0.1:6813/2328339827] exists,up d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 2026-02-21T01:47:05.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:47:05.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:47:05.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:47:05.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:521: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:47:05.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:523: TEST_divergent_2: ceph osd in osd.0 2026-02-21T01:47:05.698 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already in. 2026-02-21T01:47:05.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:521: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:47:05.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:523: TEST_divergent_2: ceph osd in osd.2 2026-02-21T01:47:05.975 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already in. 2026-02-21T01:47:05.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:526: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:47:05.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:526: TEST_divergent_2: wait_for_clean 2026-02-21T01:47:05.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:47:05.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:47:05.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:47:05.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:47:05.989 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:47:05.989 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:47:05.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:47:05.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:47:05.990 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:47:06.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:47:06.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:47:06.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:47:06.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:47:06.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:47:06.049 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:47:06.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:47:06.260 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:47:06.260 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:47:06.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:47:06.261 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:47:06.261 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:47:06.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855363 2026-02-21T01:47:06.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855363 2026-02-21T01:47:06.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855363' 2026-02-21T01:47:06.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:47:06.343 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:47:06.404 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:47:06.406 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:47:06.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:47:06.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:47:06.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:47:06.407 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:47:06.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757251 2026-02-21T01:47:06.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757251 2026-02-21T01:47:06.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855363 2-163208757251' 2026-02-21T01:47:06.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:47:06.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855363 2026-02-21T01:47:06.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:47:06.487 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:47:06.487 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855363 2026-02-21T01:47:06.488 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:47:06.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855363 2026-02-21T01:47:06.489 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 150323855363 2026-02-21T01:47:06.490 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855363' 2026-02-21T01:47:06.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:47:06.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 150323855363 2026-02-21T01:47:06.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:47:07.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:47:07.707 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:47:07.932 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-21T01:47:07.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:47:07.933 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-163208757251 2026-02-21T01:47:07.933 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:47:07.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:47:07.935 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-163208757251 2026-02-21T01:47:07.935 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:47:07.936 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757251 2026-02-21T01:47:07.937 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 163208757251' 2026-02-21T01:47:07.937 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 163208757251 2026-02-21T01:47:07.937 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:47:08.157 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-21T01:47:08.157 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:47:08.158 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:08.159 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:08.430 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:08.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:08.635 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:08.635 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:08.635 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:08.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:08.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:47:08.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:47:08.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:47:08.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:08.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:47:09.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:09.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:09.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:09.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:09.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:09.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:09.024 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:09.024 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:09.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:09.234 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:09.234 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:09.234 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:09.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:09.514 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:09.514 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:09.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:09.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:09.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:47:09.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:09.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:47:10.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:10.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:10.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:10.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:10.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:10.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:10.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:10.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:10.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:10.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:10.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:10.222 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:10.511 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:10.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:10.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:10.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:47:10.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:10.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:47:11.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:11.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:11.199 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:11.199 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:11.199 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:11.199 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:11.200 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:11.200 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:11.413 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:11.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:11.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:11.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:11.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:11.699 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:11.699 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:11.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:11.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:11.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:47:11.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:11.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:47:12.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:12.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:12.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:12.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:12.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:12.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:12.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:12.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:13.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:13.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:13.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:13.284 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:13.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:13.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:13.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:13.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:47:13.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:13.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:47:15.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:15.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:15.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:15.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:15.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:15.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:15.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:15.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:15.369 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:15.369 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:15.370 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:15.370 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:15.644 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:15.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:15.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:15.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:47:15.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:15.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:47:19.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:19.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:19.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:19.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:19.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:19.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:19.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:19.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:19.325 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:19.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:19.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:19.327 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:19.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:19.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:19.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:19.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:19.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:19.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:19.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:19.600 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:19.600 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:19.882 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:19.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:19.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:47:19.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:19.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:47:26.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:26.284 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:26.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:26.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:26.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:26.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:26.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:26.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:26.488 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:26.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:26.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:26.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:26.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:26.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:26.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:26.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:26.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:26.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:26.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:26.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:26.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:27.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:27.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:27.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:47:27.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:27.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:47:34.349 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:34.349 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:34.350 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:34.350 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:34.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:34.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:34.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:34.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:34.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:34.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:34.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:34.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:34.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:34.838 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:34.838 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:35.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:35.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:35.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:47:35.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:47:35.396 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:47:35.396 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "3f7caef3-8d3a-4b37-bd9c-73daf83baab7", 2026-02-21T01:47:35.396 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:47:35.396 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:47:35.396 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:47:35.397703+0000", 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:47:35.397 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.398 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:47:35.399 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:47:35.400 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:46:22.744835Z", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:46:22.744835Z", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:47:35.401 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.402 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:47:35.403 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:47:35.404 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 41, 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:46:22.843770+0000", 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:47:06.022580+0000", 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:47:05.075111+0000", 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:47:05.920244+0000", 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:47:35.405 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 9, 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:47:35.406 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:46:34.644629+0000", 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 1, 2026-02-21T01:47:35.407 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool", 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 0, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "off", 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 1, 2026-02-21T01:47:35.408 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 1, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 1, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 1, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.409 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "28", 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "0", 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "selfmanaged", 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 0, 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 0, 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [], 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:47:35.410 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "", 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:47:35.411 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:47:35.412 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {}, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "read_balance": { 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "score_type": "Fair distribution", 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "score_acting": 2.9900000095367432, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "score_stable": 2.9900000095367432, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_score": 0.67000001668930054, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_acting": 2, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_stable": 2, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity_weighted": 1, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity": 1, 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity_weighted": 1 2026-02-21T01:47:35.413 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "6f27f87b-8ce5-4bda-b3d5-0b765636ec02", 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 5, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 28, 2026-02-21T01:47:35.414 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 35, 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 40, 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 29, 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.415 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.416 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:47:35.417 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.418 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/3496815257", 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/3496815257", 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/3496815257", 2026-02-21T01:47:35.419 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/3496815257", 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "13b34d7e-7523-41ba-bf17-153ca2241744", 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:47:35.420 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 9, 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 22, 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 30, 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.421 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:47:35.422 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:47:35.423 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.424 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/134190718", 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/134190718", 2026-02-21T01:47:35.425 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/134190718", 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/134190718", 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74", 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:47:35.426 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 14, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 28, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 38, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 31, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.427 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:47:35.428 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.429 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:47:35.430 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/2328339827", 2026-02-21T01:47:35.431 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/2328339827", 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2328339827", 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2328339827", 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:47:00.771912+0000", 2026-02-21T01:47:35.432 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:46:26.267709+0000", 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:47:01.048202+0000", 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:47:35.433 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:46:29.813986+0000", 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:47:01.319345+0000", 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:46:32.785341+0000", 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.434 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:47:35.435 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.436 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/3496815257,v1:127.0.0.1:6803/3496815257]", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:47:35.437 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:46:23.814586Z", 2026-02-21T01:47:35.438 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/3496815257,v1:127.0.0.1:6801/3496815257]", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/3496815257,v1:127.0.0.1:6807/3496815257]", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/3496815257,v1:127.0.0.1:6805/3496815257]", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:47:35.439 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/134190718,v1:127.0.0.1:6813/134190718]", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:47:35.440 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:47:35.441 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:46:27.445252Z", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/134190718,v1:127.0.0.1:6811/134190718]", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/134190718,v1:127.0.0.1:6817/134190718]", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/134190718,v1:127.0.0.1:6815/134190718]", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:47:35.442 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/2328339827,v1:127.0.0.1:6813/2328339827]", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:47:35.443 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:47:35.444 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:46:30.294606Z", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/2328339827,v1:127.0.0.1:6811/2328339827]", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2328339827,v1:127.0.0.1:6817/2328339827]", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2328339827,v1:127.0.0.1:6815/2328339827]", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:47:35.445 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 0, 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [] 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.446 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 35 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 39 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 41, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:47:35.447 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:47:35.448 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:47:35.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:47:35.450 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:47:35.451 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.452 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:47:35.453 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:47:35.454 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:47:35.455 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 0, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:47:35.456 INFO:tasks.workunit.client.0.trial009.stderr:report 1077040062 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:46:22:843544+0000", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:47:35.457 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 1, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51200, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:47:35.458 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 100, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 100, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:47:35.459 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 16, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 16, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "up": 2, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 2, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:47:35.460 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83880, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2096, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 24, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81703, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488920, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036654080, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2146304, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 429161, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 25402, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83664070 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:47:35.461 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83880, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2096, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 24, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81703, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488920, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:47:35.462 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036654080, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2146304, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 429161, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 25402, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83664070 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.463 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51200, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 100, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 100, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:47:35.464 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1220608, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 153552, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 16, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 16, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "up": 2, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 2, 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:47:35.465 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 150323855369 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 38654705677 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 163208757257 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "num": 1 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.466 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 1, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 3, 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:47:31.266804+0000", 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "services": { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "osd": { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "daemons": { 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "summary": "", 2026-02-21T01:47:35.467 INFO:tasks.workunit.client.0.trial009.stdout: "0": { 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "2": { 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 37, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 217, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 306, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:47:35.468 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:47:35.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:47:35.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: expr 2 + 2 2026-02-21T01:47:35.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: objname=existing_4 2026-02-21T01:47:35.469 INFO:tasks.workunit.client.0.trial009.stdout:writing non-divergent object existing_4 2026-02-21T01:47:35.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:530: TEST_divergent_2: echo 'writing non-divergent object existing_4' 2026-02-21T01:47:35.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:531: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:47:35.609 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:47:35.610 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 28'100 41:130 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:47:35.610 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:47:35.610 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:47:35.610 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:47:35.618 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:532: TEST_divergent_2: rados -p test put existing_4 /tmp/tmp.ldUC5EQFKa 2026-02-21T01:47:35.647 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: rm -f /tmp/tmp.ldUC5EQFKa 2026-02-21T01:47:35.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:535: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:47:35.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:535: TEST_divergent_2: wait_for_clean 2026-02-21T01:47:35.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:47:35.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:47:35.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:47:35.650 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:47:35.650 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:47:35.650 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:47:35.651 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:47:35.651 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:47:35.651 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:47:35.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:47:35.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:47:35.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:47:35.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:47:35.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:47:35.722 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:47:35.939 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:47:35.939 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:47:35.939 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:47:35.939 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:47:35.939 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:47:35.940 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:47:36.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855371 2026-02-21T01:47:36.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855371 2026-02-21T01:47:36.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855371' 2026-02-21T01:47:36.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:47:36.021 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:47:36.079 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:47:36.081 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:47:36.081 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:47:36.081 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:47:36.081 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:47:36.081 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:47:36.159 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757259 2026-02-21T01:47:36.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757259 2026-02-21T01:47:36.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855371 2-163208757259' 2026-02-21T01:47:36.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:47:36.160 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855371 2026-02-21T01:47:36.160 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:47:36.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:47:36.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855371 2026-02-21T01:47:36.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:47:36.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855371 2026-02-21T01:47:36.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855371' 2026-02-21T01:47:36.164 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 150323855371 2026-02-21T01:47:36.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:47:36.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855369 -lt 150323855371 2026-02-21T01:47:36.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:47:37.381 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:47:37.382 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:47:37.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855371 -lt 150323855371 2026-02-21T01:47:37.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:47:37.593 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-163208757259 2026-02-21T01:47:37.593 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:47:37.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:47:37.594 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-163208757259 2026-02-21T01:47:37.594 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:47:37.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757259 2026-02-21T01:47:37.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 163208757259' 2026-02-21T01:47:37.596 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 163208757259 2026-02-21T01:47:37.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:47:37.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757259 -lt 163208757259 2026-02-21T01:47:37.824 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:47:37.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:37.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:38.108 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:47:38.108 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:38.108 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:38.109 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:38.109 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:38.109 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:38.109 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:38.109 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:38.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:38.316 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:38.316 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:38.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:38.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:38.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:47:38.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:47:38.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:47:38.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:38.590 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:47:38.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:38.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:38.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:38.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:38.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:38.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:38.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:38.693 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:38.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:38.894 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:38.894 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:38.894 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:39.170 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:39.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:39.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:39.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:47:39.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:39.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:47:39.647 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:39.647 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:39.648 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:39.648 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:39.648 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:39.648 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:39.648 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:39.649 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:39.853 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:39.853 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:39.854 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:39.854 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:40.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:40.130 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:40.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:40.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:40.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:47:40.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:40.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:47:40.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:40.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:40.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:40.817 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:40.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:40.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:40.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:40.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:41.030 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:41.031 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:41.031 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:41.031 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:41.316 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:41.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:41.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:41.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:41.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:47:41.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:41.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:47:42.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:42.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:42.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:42.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:42.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:42.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:42.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:42.405 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:42.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:42.616 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:42.616 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:42.616 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:42.900 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:42.900 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:42.901 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:43.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:43.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:43.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:47:43.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:43.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:47:44.783 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:44.783 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:44.784 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:44.785 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:44.785 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:44.785 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:44.785 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:44.785 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:45.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:45.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:45.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:45.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:45.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:45.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:45.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:45.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:45.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:47:45.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:45.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:47:48.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:48.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:48.771 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:48.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:48.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:48.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:48.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:48.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:48.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:48.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:48.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:48.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:49.270 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:49.271 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:49.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:49.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:49.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:47:49.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:49.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:47:55.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:47:55.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:47:55.962 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:47:55.962 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:47:55.962 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:47:55.962 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:47:55.963 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:47:55.963 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:47:56.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:47:56.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:47:56.194 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:47:56.194 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:47:56.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:47:56.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:47:56.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:47:56.752 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:47:56.753 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:47:56.753 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:47:56.753 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:47:56.753 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:48:04.054 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:48:04.054 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:48:04.055 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:48:04.055 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:48:04.055 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:48:04.056 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:48:04.056 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:48:04.056 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:48:04.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:48:04.261 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:48:04.261 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:48:04.261 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:48:04.536 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:48:04.537 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:48:04.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:48:04.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:48:04.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:48:04.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:48:05.088 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:48:05.088 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "3f7caef3-8d3a-4b37-bd9c-73daf83baab7", 2026-02-21T01:48:05.088 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:48:05.088 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:48:05.088 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:48:05.086451+0000", 2026-02-21T01:48:05.088 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:48:05.089 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:48:05.090 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.091 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:48:05.092 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:46:22.744835Z", 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:46:22.744835Z", 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:48:05.093 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:48:05.094 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.095 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.096 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.097 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 41, 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:46:22.843770+0000", 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:47:06.022580+0000", 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:47:05.075111+0000", 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:47:05.920244+0000", 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:48:05.098 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:48:05.099 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:48:05.099 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:48:05.099 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:48:05.099 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:48:05.099 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:48:05.099 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:48:05.100 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:48:05.100 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:48:05.100 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.100 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 9, 2026-02-21T01:48:05.100 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:48:05.100 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:48:05.101 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.102 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:48:05.102 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:48:05.102 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:46:34.644629+0000", 2026-02-21T01:48:05.102 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 1, 2026-02-21T01:48:05.102 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool", 2026-02-21T01:48:05.102 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 0, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:48:05.103 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "off", 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 1, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 1, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 1, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 1, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.104 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "28", 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "0", 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "selfmanaged", 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 0, 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 0, 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [], 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:48:05.105 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "", 2026-02-21T01:48:05.106 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:48:05.107 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {}, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "read_balance": { 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "score_type": "Fair distribution", 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "score_acting": 2.9900000095367432, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "score_stable": 2.9900000095367432, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_score": 0.67000001668930054, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_acting": 2, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_stable": 2, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity_weighted": 1, 2026-02-21T01:48:05.108 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity": 1, 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity_weighted": 1 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "6f27f87b-8ce5-4bda-b3d5-0b765636ec02", 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:48:05.109 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 5, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 28, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 35, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 40, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 29, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:48:05.110 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.111 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.112 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:48:05.113 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3496815257 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.114 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/3496815257", 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/3496815257", 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/3496815257", 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/3496815257", 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "13b34d7e-7523-41ba-bf17-153ca2241744", 2026-02-21T01:48:05.115 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 9, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 22, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 30, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.116 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.117 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.118 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.119 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.120 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 134190718 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/134190718", 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/134190718", 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/134190718", 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/134190718", 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:48:05.121 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74", 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 14, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 28, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 38, 2026-02-21T01:48:05.122 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 31, 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.123 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.124 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.125 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:48:05.126 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2328339827 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/2328339827", 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/2328339827", 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2328339827", 2026-02-21T01:48:05.127 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2328339827", 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:47:00.771912+0000", 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:48:05.128 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:46:26.267709+0000", 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:47:01.048202+0000", 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:46:29.813986+0000", 2026-02-21T01:48:05.129 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:47:01.319345+0000", 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:46:32.785341+0000", 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:48:05.130 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.131 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.132 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/3496815257,v1:127.0.0.1:6803/3496815257]", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:48:05.133 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:46:23.814586Z", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:48:05.134 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/3496815257,v1:127.0.0.1:6801/3496815257]", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/3496815257,v1:127.0.0.1:6807/3496815257]", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/3496815257,v1:127.0.0.1:6805/3496815257]", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:48:05.135 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/134190718,v1:127.0.0.1:6813/134190718]", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:48:05.136 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:48:05.137 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:46:27.445252Z", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/134190718,v1:127.0.0.1:6811/134190718]", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/134190718,v1:127.0.0.1:6817/134190718]", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/134190718,v1:127.0.0.1:6815/134190718]", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:48:05.138 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/2328339827,v1:127.0.0.1:6813/2328339827]", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:48:05.139 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:48:05.140 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:46:30.294606Z", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/2328339827,v1:127.0.0.1:6811/2328339827]", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2328339827,v1:127.0.0.1:6817/2328339827]", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2328339827,v1:127.0.0.1:6815/2328339827]", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:48:05.141 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 0, 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [] 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 35 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 39 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 41, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:48:05.144 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:48:05.145 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:48:05.146 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:48:05.147 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:48:05.148 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.149 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.150 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:48:05.151 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 0, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:48:05.152 INFO:tasks.workunit.client.0.trial009.stderr:report 2413091238 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:46:22:843544+0000", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:48:05.153 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 1, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51688, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:48:05.154 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:48:05.155 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 17, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 17, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "up": 2, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 2, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:48:05.156 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83880, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2096, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 24, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81703, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488920, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036654080, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2146304, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 430137, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 25402, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83664070 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:48:05.157 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83880, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2096, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 24, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81703, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488920, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036654080, 2026-02-21T01:48:05.158 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2146304, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 430137, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 25402, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83664070 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:48:05.159 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51688, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:48:05.160 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1220608, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 154528, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 17, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 17, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "up": 2, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 2, 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:48:05.161 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 150323855376 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 38654705677 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 163208757264 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "num": 1 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.162 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 1, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 3, 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:47:31.266804+0000", 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "services": { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "osd": { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "daemons": { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "summary": "", 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "0": { 2026-02-21T01:48:05.163 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "2": { 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 51, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 281, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 384, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:48:05.164 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:48:05.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:48:05.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: echo 'delay recovery' 2026-02-21T01:48:05.165 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery 2026-02-21T01:48:05.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:539: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:05.315 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:05.315 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 41'101 41:131 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:48:05.315 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:05.315 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:05.315 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:05.324 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:48:05.325 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: get_asok_path osd.0 2026-02-21T01:48:05.325 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:48:05.325 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:48:05.326 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:48:05.326 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:05.326 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:05.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:48:05.326 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: CEPH_ARGS= 2026-02-21T01:48:05.326 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok set_recovery_delay 100000 2026-02-21T01:48:05.382 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:48:05.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: for i in $non_divergent 2026-02-21T01:48:05.388 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: get_asok_path osd.2 2026-02-21T01:48:05.388 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:48:05.388 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:48:05.389 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:48:05.389 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:05.389 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:05.389 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:48:05.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: CEPH_ARGS= 2026-02-21T01:48:05.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:542: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok set_recovery_delay 100000 2026-02-21T01:48:05.444 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:48:05.450 INFO:tasks.workunit.client.0.trial009.stdout:revive divergent 1 2026-02-21T01:48:05.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:546: TEST_divergent_2: echo 'revive divergent 1' 2026-02-21T01:48:05.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:547: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:05.664 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:05.664 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 41'101 41:131 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:48:05.665 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:05.665 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:05.665 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:05.673 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:548: TEST_divergent_2: ceph osd set noup 2026-02-21T01:48:06.003 INFO:tasks.workunit.client.0.trial009.stderr:noup is set 2026-02-21T01:48:06.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:549: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-21T01:48:06.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:48:06.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:48:06.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:48:06.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:48:06.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:48:06.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:48:06.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:48:06.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:48:06.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:48:06.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:48:06.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:48:06.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:48:06.022 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:48:06.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:48:06.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:48:06.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:48:06.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:48:06.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:48:06.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:48:06.042 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:06.036+0000 7f2333f83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:06.044 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:06.040+0000 7f2333f83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:06.045 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:06.040+0000 7f2333f83a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:06.236 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-21T01:48:06.236 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:550: TEST_divergent_2: sleep 5 2026-02-21T01:48:06.457 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:06.452+0000 7f2333f83a40 -1 Falling back to public interface 2026-02-21T01:48:07.150 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:07.144+0000 7f2333f83a40 -1 osd.1 28 log_to_monitors true 2026-02-21T01:48:11.237 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery divergent 2026-02-21T01:48:11.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: echo 'delay recovery divergent' 2026-02-21T01:48:11.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:553: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:11.439 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:11.439 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 41'101 44:137 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:48:11.440 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:11.440 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:11.440 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:11.450 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: get_asok_path osd.1 2026-02-21T01:48:11.450 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:48:11.450 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:48:11.450 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:48:11.450 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:11.450 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:11.451 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:48:11.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: CEPH_ARGS= 2026-02-21T01:48:11.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok set_recovery_delay 100000 2026-02-21T01:48:11.506 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:48:11.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:556: TEST_divergent_2: ceph osd unset noup 2026-02-21T01:48:11.825 INFO:tasks.workunit.client.0.trial009.stderr:noup is unset 2026-02-21T01:48:11.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:558: TEST_divergent_2: wait_for_osd up 0 2026-02-21T01:48:11.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:11.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:48:11.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:11.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:11.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:11.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:11.837 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:11.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:11.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:48:12.056 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 35 up_thru 40 down_at 29 last_clean_interval [5,28) [v2:127.0.0.1:6800/3496815257,v1:127.0.0.1:6801/3496815257] [v2:127.0.0.1:6802/3496815257,v1:127.0.0.1:6803/3496815257] exists,up 6f27f87b-8ce5-4bda-b3d5-0b765636ec02 2026-02-21T01:48:12.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:12.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:12.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:559: TEST_divergent_2: wait_for_osd up 1 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:12.058 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:12.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:12.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:48:12.271 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/2375153530,v1:127.0.0.1:6819/2375153530] [v2:127.0.0.1:6820/2375153530,v1:127.0.0.1:6821/2375153530] exists,up 13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:48:12.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:12.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:12.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:12.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:560: TEST_divergent_2: wait_for_osd up 2 2026-02-21T01:48:12.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:12.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:48:12.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:12.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:12.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:12.273 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:12.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:48:12.484 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 38 up_thru 0 down_at 31 last_clean_interval [14,28) [v2:127.0.0.1:6810/2328339827,v1:127.0.0.1:6811/2328339827] [v2:127.0.0.1:6812/2328339827,v1:127.0.0.1:6813/2328339827] exists,up d0f8a3b7-f946-4bdf-a5ab-456fa7aafb74 2026-02-21T01:48:12.484 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:12.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:12.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:12.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:562: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:12.688 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:12.688 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 41'101 44:137 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:48:12.689 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:12.689 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:12.689 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:12.697 INFO:tasks.workunit.client.0.trial009.stdout:wait for peering 2026-02-21T01:48:12.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:563: TEST_divergent_2: echo 'wait for peering' 2026-02-21T01:48:12.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:564: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:12.919 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:12.919 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 41'101 44:137 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:48:12.919 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:12.919 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:12.919 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:12.928 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:565: TEST_divergent_2: rados -p test put foo /tmp/tmp.T4YSTHp4YX 2026-02-21T01:48:12.959 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:566: TEST_divergent_2: rm -f /tmp/tmp.T4YSTHp4YX 2026-02-21T01:48:12.960 INFO:tasks.workunit.client.0.trial009.stdout:killing divergent 1 2026-02-21T01:48:12.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:570: TEST_divergent_2: echo 'killing divergent 1' 2026-02-21T01:48:12.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:571: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:13.160 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:13.161 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51688 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:47:06.025354+0000 41'101 44:137 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T09:35:20.862889+0000 0 0 2026-02-21T01:48:13.161 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:13.161 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:13.161 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:572: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd.1 2026-02-21T01:48:13.171 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:48:13.171 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:48:13.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:48:13.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:48:13.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:48:13.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:48:13.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:575: TEST_divergent_2: expfile=td/divergent-priors/exp.20522.out 2026-02-21T01:48:13.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:576: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20522.out 2026-02-21T01:48:13.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:48:13.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:48:13.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T01:48:13.281 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:48:13.281 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-21T01:48:13.281 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20522.out 2026-02-21T01:48:13.477 INFO:tasks.workunit.client.0.trial009.stderr:Exporting 1.0 info 1.0( v 47'102 lc 0'0 (28'90,47'102] local-lis/les=46/47 n=101 ec=22/22 lis/c=46/22 les/c/f=47/23/0 sis=46) 2026-02-21T01:48:13.485 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-21T01:48:13.485 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-21T01:48:13.485 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:091f9f1d:::existing_94:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:0a865a93:::existing_54:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:0fed513c:::existing_41:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:13b9e0b8:::existing_92:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:169157fa:::existing_81:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:17731382:::existing_58:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:18a45a39:::existing_43:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:1c160f01:::existing_66:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:1dd6c249:::existing_97:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:227e0c77:::existing_7:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:23e1b063:::existing_59:head# 2026-02-21T01:48:13.486 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:28566edb:::existing_78:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:2a101182:::existing_20:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:2d1519da:::existing_96:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:3078b87a:::existing_86:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:30bde01b:::existing_15:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:317a7e19:::existing_56:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:38b9c7fa:::existing_33:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:3e16d84b:::existing_48:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:45fe8203:::existing_75:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:4e82c9ba:::existing_50:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:5298dcdb:::existing_57:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:52ebaa44:::existing_32:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:53588912:::existing_9:head# 2026-02-21T01:48:13.487 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:5b7e2de6:::existing_84:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:5da0255c:::existing_13:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:5da2dab8:::existing_53:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:5ee3514b:::existing_30:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:602f83fe:::foo:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:60426cc7:::existing_62:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:628918ca:::existing_79:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:63f8ab2d:::existing_77:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:67e1780f:::existing_27:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:68143ca1:::existing_71:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:6bac9509:::existing_29:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:6bbec798:::existing_45:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:6cb23a40:::existing_65:head# 2026-02-21T01:48:13.488 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:7055ec5e:::existing_83:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:70a54e2b:::existing_87:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:750ace21:::existing_19:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:78623b1d:::existing_6:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:79881152:::existing_63:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:7a80c0d5:::existing_99:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:81490677:::existing_76:head# 2026-02-21T01:48:13.489 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:85ec8e5d:::existing_85:head# 2026-02-21T01:48:13.498 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:8e2abbf0:::existing_49:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:8ee0fe0f:::existing_52:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:923c5d4d:::existing_5:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:92731e71:::existing_89:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:928205cf:::existing_82:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:93be2267:::existing_24:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:99ceda2c:::existing_44:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:9dd917ee:::existing_26:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a28805a6:::existing_72:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a294a8f6:::existing_16:head# 2026-02-21T01:48:13.499 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a2f5473a:::existing_42:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a504a533:::existing_90:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a691337e:::existing_61:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a6e8d7ac:::existing_28:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:a9fde86e:::existing_18:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:aa309faa:::existing_68:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:ae40b0ed:::existing_31:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:b279b1fe:::existing_10:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:b343e277:::existing_88:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:b7130e90:::existing_80:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:bc6f7309:::existing_70:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:be8489b7:::existing_38:head# 2026-02-21T01:48:13.500 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:beb4e5e3:::existing_47:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:bebec877:::existing_74:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:c04347d8:::existing_60:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:c156033e:::existing_100:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:c19f5a8e:::existing_8:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:c3e6ebe4:::existing_17:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:c6c5a7c2:::existing_39:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:cc1d0abf:::existing_95:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:ccb3865b:::existing_73:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:cd84e19e:::existing_40:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:d2cd64ce:::existing_37:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:d41a25c9:::existing_22:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:d7f088da:::existing_91:head# 2026-02-21T01:48:13.501 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:d9cbd1d6:::existing_64:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:dcc4ff82:::existing_98:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:dd677bd3:::existing_35:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:dd6d90e7:::existing_46:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:e0e22077:::existing_21:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:e255b710:::existing_55:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:eff1d5db:::existing_14:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:f1b156bf:::existing_23:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:f5ce4199:::existing_34:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:f6809456:::existing_11:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:f699a2a9:::existing_67:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:f9baac43:::existing_25:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:fb76688e:::existing_51:head# 2026-02-21T01:48:13.502 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:fbb92eec:::existing_69:head# 2026-02-21T01:48:13.503 INFO:tasks.workunit.client.0.trial009.stderr:Export successful 2026-02-21T01:48:13.503 INFO:tasks.workunit.client.0.trial009.stdout: marking collection for removal 2026-02-21T01:48:13.503 INFO:tasks.workunit.client.0.trial009.stdout:setting '_remove' omap key 2026-02-21T01:48:13.503 INFO:tasks.workunit.client.0.trial009.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-21T01:48:13.504 INFO:tasks.workunit.client.0.trial009.stderr:Remove successful 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:577: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.20522.out 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-21T01:48:13.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.20522.out 2026-02-21T01:48:13.828 INFO:tasks.workunit.client.0.trial009.stdout:get_pg_num_history pg_num_history pg_num_history(e47 pg_nums {1={22=1}} deleted_pools ) 2026-02-21T01:48:13.828 INFO:tasks.workunit.client.0.trial009.stdout:Importing pgid 1.0 2026-02-21T01:48:13.828 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-21T01:48:13.828 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.828 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-21T01:48:13.828 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:091f9f1d:::existing_94:head# 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:0a865a93:::existing_54:head# 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.829 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:0fed513c:::existing_41:head# 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:13b9e0b8:::existing_92:head# 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:169157fa:::existing_81:head# 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:17731382:::existing_58:head# 2026-02-21T01:48:13.830 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:18a45a39:::existing_43:head# 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:1c160f01:::existing_66:head# 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:1dd6c249:::existing_97:head# 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.831 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:227e0c77:::existing_7:head# 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:23e1b063:::existing_59:head# 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:28566edb:::existing_78:head# 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:2a101182:::existing_20:head# 2026-02-21T01:48:13.832 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:2d1519da:::existing_96:head# 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:3078b87a:::existing_86:head# 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:30bde01b:::existing_15:head# 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.833 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:317a7e19:::existing_56:head# 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:38b9c7fa:::existing_33:head# 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:3e16d84b:::existing_48:head# 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:45fe8203:::existing_75:head# 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.834 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:4e82c9ba:::existing_50:head# 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:5298dcdb:::existing_57:head# 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:52ebaa44:::existing_32:head# 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:53588912:::existing_9:head# 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.835 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:5b7e2de6:::existing_84:head# 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:5da0255c:::existing_13:head# 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:5da2dab8:::existing_53:head# 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:5ee3514b:::existing_30:head# 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.836 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:602f83fe:::foo:head# 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:60426cc7:::existing_62:head# 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:628918ca:::existing_79:head# 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:63f8ab2d:::existing_77:head# 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.837 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:67e1780f:::existing_27:head# 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:68143ca1:::existing_71:head# 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:6bac9509:::existing_29:head# 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:6bbec798:::existing_45:head# 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:6cb23a40:::existing_65:head# 2026-02-21T01:48:13.838 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:7055ec5e:::existing_83:head# 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:70a54e2b:::existing_87:head# 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:750ace21:::existing_19:head# 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:78623b1d:::existing_6:head# 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.839 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:79881152:::existing_63:head# 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:7a80c0d5:::existing_99:head# 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:81490677:::existing_76:head# 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:85ec8e5d:::existing_85:head# 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:8e2abbf0:::existing_49:head# 2026-02-21T01:48:13.840 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:8ee0fe0f:::existing_52:head# 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:923c5d4d:::existing_5:head# 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:92731e71:::existing_89:head# 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:928205cf:::existing_82:head# 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:93be2267:::existing_24:head# 2026-02-21T01:48:13.841 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:99ceda2c:::existing_44:head# 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:9dd917ee:::existing_26:head# 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a28805a6:::existing_72:head# 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a294a8f6:::existing_16:head# 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.842 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a2f5473a:::existing_42:head# 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a504a533:::existing_90:head# 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a691337e:::existing_61:head# 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a6e8d7ac:::existing_28:head# 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:a9fde86e:::existing_18:head# 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.843 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:aa309faa:::existing_68:head# 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:ae40b0ed:::existing_31:head# 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:b279b1fe:::existing_10:head# 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:b343e277:::existing_88:head# 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:b7130e90:::existing_80:head# 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:bc6f7309:::existing_70:head# 2026-02-21T01:48:13.844 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:be8489b7:::existing_38:head# 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:beb4e5e3:::existing_47:head# 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:bebec877:::existing_74:head# 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:c04347d8:::existing_60:head# 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:c156033e:::existing_100:head# 2026-02-21T01:48:13.845 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:c19f5a8e:::existing_8:head# 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:c3e6ebe4:::existing_17:head# 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:c6c5a7c2:::existing_39:head# 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:cc1d0abf:::existing_95:head# 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:ccb3865b:::existing_73:head# 2026-02-21T01:48:13.846 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:cd84e19e:::existing_40:head# 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:d2cd64ce:::existing_37:head# 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:d41a25c9:::existing_22:head# 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:d7f088da:::existing_91:head# 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:d9cbd1d6:::existing_64:head# 2026-02-21T01:48:13.847 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:dcc4ff82:::existing_98:head# 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:dd677bd3:::existing_35:head# 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:dd6d90e7:::existing_46:head# 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:e0e22077:::existing_21:head# 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:e255b710:::existing_55:head# 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.848 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:eff1d5db:::existing_14:head# 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:f1b156bf:::existing_23:head# 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:f5ce4199:::existing_34:head# 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:f6809456:::existing_11:head# 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:f699a2a9:::existing_67:head# 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:f9baac43:::existing_25:head# 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.849 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:fb76688e:::existing_51:head# 2026-02-21T01:48:13.850 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.850 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:fbb92eec:::existing_69:head# 2026-02-21T01:48:13.850 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:48:13.850 INFO:tasks.workunit.client.0.trial009.stdout:write_pg epoch 47 info 1.0( v 47'102 lc 0'0 (28'90,47'102] local-lis/les=46/47 n=101 ec=22/22 lis/c=46/22 les/c/f=47/23/0 sis=46) 2026-02-21T01:48:13.850 INFO:tasks.workunit.client.0.trial009.stdout:Import successful 2026-02-21T01:48:13.917 INFO:tasks.workunit.client.0.trial009.stdout:reviving divergent 1 2026-02-21T01:48:13.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:579: TEST_divergent_2: echo 'reviving divergent 1' 2026-02-21T01:48:13.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:580: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:14.113 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:14.113 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 4 4 0 0 51688 0 0 11 90 11 activating+degraded 2026-02-21T01:48:11.930081+0000 41'101 47:134 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:48:14.113 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:14.113 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:14.113 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:581: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:48:14.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:14.124 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:14.125 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:14.125 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:14.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:14.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:48:14.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:48:14.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:48:14.127 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:48:14.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:48:14.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:48:14.129 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:48:14.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:48:14.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:48:14.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:48:14.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:48:14.150 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:14.143+0000 7f115cb11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:14.151 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:14.143+0000 7f115cb11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:14.152 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:14.143+0000 7f115cb11a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:14.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:14.344 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:14.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:14.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:48:14.522 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:14.515+0000 7f115cb11a40 -1 Falling back to public interface 2026-02-21T01:48:14.552 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/2375153530,v1:127.0.0.1:6819/2375153530] [v2:127.0.0.1:6820/2375153530,v1:127.0.0.1:6821/2375153530] exists,up 13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: wait_for_osd up 1 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:48:14.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:14.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:14.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:14.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:14.555 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:14.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:14.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:48:14.759 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/2375153530,v1:127.0.0.1:6819/2375153530] [v2:127.0.0.1:6820/2375153530,v1:127.0.0.1:6821/2375153530] exists,up 13b34d7e-7523-41ba-bf17-153ca2241744 2026-02-21T01:48:14.760 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:14.760 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:14.760 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:14.760 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:584: TEST_divergent_2: sleep 20 2026-02-21T01:48:15.237 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:15.231+0000 7f115cb11a40 -1 osd.1 47 log_to_monitors true 2026-02-21T01:48:34.763 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: get_asok_path osd.1 2026-02-21T01:48:34.764 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:48:34.764 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:48:34.764 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:48:34.764 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:34.764 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:34.765 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:48:34.765 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: CEPH_ARGS= 2026-02-21T01:48:34.765 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok dump_ops_in_flight 2026-02-21T01:48:34.826 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:48:34.826 INFO:tasks.workunit.client.0.trial009.stdout: "ops": [], 2026-02-21T01:48:34.826 INFO:tasks.workunit.client.0.trial009.stdout: "num_ops": 0 2026-02-21T01:48:34.826 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:48:34.832 INFO:tasks.workunit.client.0.trial009.stdout:allowing recovery 2026-02-21T01:48:34.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: echo 'allowing recovery' 2026-02-21T01:48:34.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:588: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:35.043 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:35.043 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 52200 0 0 12 90 12 active+clean 2026-02-21T01:48:15.912917+0000 47'102 51:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:53:16.242086+0000 0 0 2026-02-21T01:48:35.043 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:35.043 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:35.043 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:35.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:590: TEST_divergent_2: for i in $osds 2026-02-21T01:48:35.052 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-21T01:48:35.139 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:48:35.146 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:590: TEST_divergent_2: for i in $osds 2026-02-21T01:48:35.146 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-21T01:48:35.227 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:48:35.234 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:590: TEST_divergent_2: for i in $osds 2026-02-21T01:48:35.234 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-21T01:48:35.313 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:48:35.321 INFO:tasks.workunit.client.0.trial009.stdout:reading divergent objects 2026-02-21T01:48:35.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:595: TEST_divergent_2: echo 'reading divergent objects' 2026-02-21T01:48:35.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: ceph pg dump pgs 2026-02-21T01:48:35.526 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:48:35.526 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 52200 0 0 12 90 12 active+clean 2026-02-21T01:48:15.912917+0000 47'102 51:162 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:46:34.666409+0000 0'0 2026-02-21T01:46:34.666409+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:53:16.242086+0000 0 0 2026-02-21T01:48:35.526 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:48:35.526 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:48:35.527 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:48:35.535 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: expr 2 + 2 2026-02-21T01:48:35.537 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: seq 1 4 2026-02-21T01:48:35.538 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:48:35.538 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_1 td/divergent-priors/existing 2026-02-21T01:48:35.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:48:35.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_2 td/divergent-priors/existing 2026-02-21T01:48:35.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:48:35.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_3 td/divergent-priors/existing 2026-02-21T01:48:35.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:597: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:48:35.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:599: TEST_divergent_2: rados -p test get existing_4 td/divergent-priors/existing 2026-02-21T01:48:35.644 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:601: TEST_divergent_2: seq 1 2 2026-02-21T01:48:35.645 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:601: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:48:35.645 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:603: TEST_divergent_2: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-21T01:48:35.664 INFO:tasks.workunit.client.0.trial009.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-21T01:48:35.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:601: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:48:35.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:603: TEST_divergent_2: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-21T01:48:35.691 INFO:tasks.workunit.client.0.trial009.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-21T01:48:35.693 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:605: TEST_divergent_2: rm -f td/divergent-priors/existing 2026-02-21T01:48:35.695 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:607: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-21T01:48:35.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:607: TEST_divergent_2: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 28'106 (28'4) delete 1:70b3bf12:::existing_4:head by client.4549.0:1 2026-02-21T01:46:50.007558+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: keeping 28'106 (28'4) delete 1:70b3bf12:::existing_4:head by client.4549.0:1 2026-02-21T01:46:50.007558+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 28'4 first_divergent_update: 28'106 last_divergent_update: 28'106 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: more recent entry found: 41'101 (28'4) modify 1:70b3bf12:::existing_4:head by client.4658.0:1 2026-02-21T01:47:35.646691+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false, already merged 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 28'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4536.0:1 2026-02-21T01:46:50.000121+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: keeping 28'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4536.0:1 2026-02-21T01:46:50.000121+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 28'101 last_divergent_update: 28'101 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-21T01:48:35.699 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4548.0:1 2026-02-21T01:46:50.006661+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: keeping 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4548.0:1 2026-02-21T01:46:50.006661+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 28'3 first_divergent_update: 28'105 last_divergent_update: 28'105 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 28'106 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 28'105 (28'3) delete 1:8881e6d8:::existing_3:head by client.4548.0:1 2026-02-21T01:46:50.006661+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 28'3 <= info.log_tail 28'90 2026-02-21T01:48:35.700 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4543.0:1 2026-02-21T01:46:50.003052+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: keeping 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4543.0:1 2026-02-21T01:46:50.003052+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 28'2 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 28'106 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 28'104 (28'2) modify 1:97db2a0f:::existing_2:head by client.4543.0:1 2026-02-21T01:46:50.003052+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 28'2 <= info.log_tail 28'90 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4542.0:1 2026-02-21T01:46:50.002765+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:48:35.701 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: keeping 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4542.0:1 2026-02-21T01:46:50.002765+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:48:35.702 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:48:35.702 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 28'1 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-21T01:48:35.702 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-21T01:48:35.702 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:48:35.702 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 28'106 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 28'103 (28'1) modify 1:d858c035:::existing_1:head by client.4542.0:1 2026-02-21T01:46:50.002765+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 28'1 <= info.log_tail 28'90 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 28'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4537.0:1 2026-02-21T01:46:50.000692+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 20 _merge_object_divergent_entries: keeping 28'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4537.0:1 2026-02-21T01:46:50.000692+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 28'102 last_divergent_update: 28'102 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-21T01:48:35.703 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:48:11.867+0000 7f2311700640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-21T01:48:35.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:609: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-21T01:48:35.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:609: TEST_divergent_2: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:48:35.705 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:615: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-21T01:48:35.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:615: TEST_divergent_2: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:48:35.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:620: TEST_divergent_2: echo success 2026-02-21T01:48:35.709 INFO:tasks.workunit.client.0.trial009.stdout:success 2026-02-21T01:48:35.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:622: TEST_divergent_2: rm td/divergent-priors/td/divergent-priors/exp.20522.out 2026-02-21T01:48:35.711 INFO:tasks.workunit.client.0.trial009.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.20522.out': No such file or directory 2026-02-21T01:48:35.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:623: TEST_divergent_2: delete_pool test 2026-02-21T01:48:35.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-21T01:48:35.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-21T01:48:35.967 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' does not exist 2026-02-21T01:48:35.976 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:624: TEST_divergent_2: kill_daemons td/divergent-priors 2026-02-21T01:48:35.977 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:48:35.977 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:48:35.977 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:48:35.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:48:35.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:48:41.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:48:41.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-21T01:48:41.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:48:41.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:48:41.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:48:41.295 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:48:41.296 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:48:41.297 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:48:41.297 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:48:41.297 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:48:41.301 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:48:41.302 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:48:41.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:48:41.303 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:48:41.304 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:48:41.304 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:48:41.305 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:48:41.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:48:41.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:48:41.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:48:41.307 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:48:41.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:48:41.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:48:41.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:48:41.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:48:41.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.329 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-21T01:48:41.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:48:41.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:48:41.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:48:41.334 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:48:41.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:48:41.335 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:48:41.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:48:41.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:48:41.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:48:41.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:48:41.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:48:41.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:48:41.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:48:41.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:48:41.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:48:41.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:48:41.341 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:48:41.341 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.341 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:48:41.342 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:48:41.342 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:48:41.342 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-21T01:48:41.344 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:48:41.344 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.344 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.20522 2026-02-21T01:48:41.346 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:48:41.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T01:48:41.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:48:41.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-21T01:48:41.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_3 td/divergent-priors 2026-02-21T01:48:41.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:630: TEST_divergent_3: local dir=td/divergent-priors 2026-02-21T01:48:41.347 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: file_with_random_data 2026-02-21T01:48:41.348 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=512 2026-02-21T01:48:41.348 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:48:41.348 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.o88sKAF9En 2026-02-21T01:48:41.348 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.o88sKAF9En bs=512 count=1 2026-02-21T01:48:41.349 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:48:41.350 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:48:41.350 INFO:tasks.workunit.client.0.trial009.stderr:512 bytes copied, 5.7245e-05 s, 8.9 MB/s 2026-02-21T01:48:41.350 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.o88sKAF9En 2026-02-21T01:48:41.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: local dummyfile=/tmp/tmp.o88sKAF9En 2026-02-21T01:48:41.350 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: file_with_random_data 1000 2026-02-21T01:48:41.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=1000 2026-02-21T01:48:41.351 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:48:41.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.clOFspByw9 2026-02-21T01:48:41.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.clOFspByw9 bs=1000 count=1 2026-02-21T01:48:41.353 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:48:41.353 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:48:41.353 INFO:tasks.workunit.client.0.trial009.stderr:1000 bytes (1.0 kB) copied, 8.0086e-05 s, 12.5 MB/s 2026-02-21T01:48:41.353 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.clOFspByw9 2026-02-21T01:48:41.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: local dummyfile2=/tmp/tmp.clOFspByw9 2026-02-21T01:48:41.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:636: TEST_divergent_3: local num_osds=3 2026-02-21T01:48:41.354 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: expr 3 - 1 2026-02-21T01:48:41.355 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: seq 0 2 2026-02-21T01:48:41.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: local 'osds=0 2026-02-21T01:48:41.356 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: run_mon td/divergent-priors a 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:48:41.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-21T01:48:41.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-21T01:48:41.380 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:48:41.380 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:41.380 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:41.380 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:41.380 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.381 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.381 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:41.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:48:41.411 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:48:41.412 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:48:41.412 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:48:41.412 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:48:41.412 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:48:41.412 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:48:41.413 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:48:41.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get fsid 2026-02-21T01:48:41.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:48:41.469 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:48:41.470 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:48:41.470 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:48:41.470 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:48:41.471 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:48:41.471 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:48:41.471 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:48:41.471 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:48:41.471 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.472 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.472 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:48:41.472 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:48:41.472 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get mon_host 2026-02-21T01:48:41.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:639: TEST_divergent_3: run_mgr td/divergent-priors x 2026-02-21T01:48:41.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-21T01:48:41.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:48:41.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:48:41.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:48:41.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-21T01:48:41.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:41.640 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:48:41.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:48:41.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: for i in $osds 2026-02-21T01:48:41.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:642: TEST_divergent_3: run_osd td/divergent-priors 0 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:48:41.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:41.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:48:41.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:48:41.667 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:48:41.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b6a9008d-1450-4e80-978d-4c4a7e607896 2026-02-21T01:48:41.667 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 b6a9008d-1450-4e80-978d-4c4a7e607896 2026-02-21T01:48:41.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b6a9008d-1450-4e80-978d-4c4a7e607896' 2026-02-21T01:48:41.668 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:48:41.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD5DplpVyUqKBAAm/AloHr0P1Lb035wNPBMVA== 2026-02-21T01:48:41.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD5DplpVyUqKBAAm/AloHr0P1Lb035wNPBMVA=="}' 2026-02-21T01:48:41.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b6a9008d-1450-4e80-978d-4c4a7e607896 -i td/divergent-priors/0/new.json 2026-02-21T01:48:41.794 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:41.802 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-21T01:48:41.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD5DplpVyUqKBAAm/AloHr0P1Lb035wNPBMVA== --osd-uuid b6a9008d-1450-4e80-978d-4c4a7e607896 2026-02-21T01:48:41.822 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:41.817+0000 7f8e2be5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:41.824 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:41.817+0000 7f8e2be5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:41.825 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:41.817+0000 7f8e2be5aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:41.825 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:41.817+0000 7f8e2be5aa40 -1 bdev(0x56458c8cec00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:48:41.825 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:41.817+0000 7f8e2be5aa40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-21T01:48:42.586 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-21T01:48:42.586 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:48:42.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:48:42.588 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:48:42.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:48:42.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:48:42.712 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:48:42.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:48:42.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:48:42.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:48:42.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:48:42.730 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:42.725+0000 7fab69d6aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:42.732 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:42.725+0000 7fab69d6aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:42.733 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:42.725+0000 7fab69d6aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:42.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:48:42.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:42.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:48:42.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:42.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:42.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:42.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:42.841 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:42.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:42.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:48:42.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:48:43.143 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:43.137+0000 7fab69d6aa40 -1 Falling back to public interface 2026-02-21T01:48:43.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:48:43.956 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:43.956 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:48:43.956 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:48:43.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:43.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:48:43.982 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:43.977+0000 7fab69d6aa40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:48:44.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:48:45.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:48:45.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:45.177 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:48:45.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:48:45.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:45.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:48:45.385 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3000301575,v1:127.0.0.1:6801/3000301575] [v2:127.0.0.1:6802/3000301575,v1:127.0.0.1:6803/3000301575] exists,up b6a9008d-1450-4e80-978d-4c4a7e607896 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: for i in $osds 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:642: TEST_divergent_3: run_osd td/divergent-priors 1 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:48:45.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:48:45.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:48:45.391 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:48:45.392 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:48:45.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:48:45.393 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:48:45.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 130a8455-684c-4f56-9f61-eae6d72f68fc' 2026-02-21T01:48:45.393 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:48:45.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD9DplpxufIFxAAUWMOSCMGQsEHcuZTKHcFEw== 2026-02-21T01:48:45.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD9DplpxufIFxAAUWMOSCMGQsEHcuZTKHcFEw=="}' 2026-02-21T01:48:45.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 130a8455-684c-4f56-9f61-eae6d72f68fc -i td/divergent-priors/1/new.json 2026-02-21T01:48:45.617 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:48:45.626 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-21T01:48:45.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD9DplpxufIFxAAUWMOSCMGQsEHcuZTKHcFEw== --osd-uuid 130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:48:45.646 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:45.641+0000 7f75ab61ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:45.647 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:45.641+0000 7f75ab61ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:45.649 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:45.641+0000 7f75ab61ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:45.650 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:45.641+0000 7f75ab61ea40 -1 bdev(0x555bdfeb8c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:48:45.650 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:45.641+0000 7f75ab61ea40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-21T01:48:46.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-21T01:48:46.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:48:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:48:46.310 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:48:46.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:48:46.587 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:48:46.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:48:46.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:48:46.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:48:46.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:48:46.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:48:46.607 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:46.601+0000 7f66b8bc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:46.608 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:46.601+0000 7f66b8bc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:46.609 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:46.601+0000 7f66b8bc3a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:46.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:48:46.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:46.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:48:47.014 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:48:47.015 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:47.009+0000 7f66b8bc3a40 -1 Falling back to public interface 2026-02-21T01:48:47.651 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:47.645+0000 7f66b8bc3a40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:48:48.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:48:48.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:48.015 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:48:48.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:48:48.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:48.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:48:48.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:48:49.226 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:48:49.226 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:49.226 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:48:49.227 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:48:49.228 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:49.228 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:48:49.442 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1003373808,v1:127.0.0.1:6811/1003373808] [v2:127.0.0.1:6812/1003373808,v1:127.0.0.1:6813/1003373808] exists,up 130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:48:49.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: for i in $osds 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:642: TEST_divergent_3: run_osd td/divergent-priors 2 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:48:49.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:48:49.451 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:48:49.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:48:49.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:48:49.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:48:49.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:48:49.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:48:49.453 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:48:49.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 2026-02-21T01:48:49.454 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 2026-02-21T01:48:49.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821' 2026-02-21T01:48:49.455 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:48:49.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQABD5lptMBWGxAAuiZ0WjBOqoA/1AQJdcOokw== 2026-02-21T01:48:49.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQABD5lptMBWGxAAuiZ0WjBOqoA/1AQJdcOokw=="}' 2026-02-21T01:48:49.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 -i td/divergent-priors/2/new.json 2026-02-21T01:48:49.687 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:48:49.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-21T01:48:49.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQABD5lptMBWGxAAuiZ0WjBOqoA/1AQJdcOokw== --osd-uuid 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 2026-02-21T01:48:49.718 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:49.713+0000 7fef85ed5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:49.719 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:49.713+0000 7fef85ed5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:49.720 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:49.713+0000 7fef85ed5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:49.721 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:49.713+0000 7fef85ed5a40 -1 bdev(0x56050ae22c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:48:49.721 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:49.713+0000 7fef85ed5a40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-21T01:48:50.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-21T01:48:50.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:48:50.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:48:50.338 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:48:50.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:48:50.638 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:48:50.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:48:50.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:48:50.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:48:50.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:48:50.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:48:50.669 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:50.661+0000 7f19b23bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:50.671 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:50.665+0000 7f19b23bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:50.672 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:50.665+0000 7f19b23bba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:50.847 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:48:50.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:48:50.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:50.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:48:51.035 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:51.029+0000 7f19b23bba40 -1 Falling back to public interface 2026-02-21T01:48:51.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:48:51.736 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:48:51.729+0000 7f19b23bba40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:48:52.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:48:52.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:52.076 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:48:52.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:48:52.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:52.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:48:52.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:48:53.293 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:48:53.293 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:48:53.294 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:48:53.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:48:53.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:48:53.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:48:53.507 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1054613536,v1:127.0.0.1:6819/1054613536] [v2:127.0.0.1:6820/1054613536,v1:127.0.0.1:6821/1054613536] exists,up 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 2026-02-21T01:48:53.508 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:48:53.508 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:48:53.508 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:48:53.508 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:645: TEST_divergent_3: ceph osd set noout 2026-02-21T01:48:53.800 INFO:tasks.workunit.client.0.trial009.stderr:noout is set 2026-02-21T01:48:53.812 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:646: TEST_divergent_3: ceph osd set noin 2026-02-21T01:48:54.107 INFO:tasks.workunit.client.0.trial009.stderr:noin is set 2026-02-21T01:48:54.119 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:647: TEST_divergent_3: ceph osd set nodown 2026-02-21T01:48:54.412 INFO:tasks.workunit.client.0.trial009.stderr:nodown is set 2026-02-21T01:48:54.423 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:648: TEST_divergent_3: create_pool test 1 1 2026-02-21T01:48:54.423 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 2026-02-21T01:48:54.666 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' already exists 2026-02-21T01:48:54.676 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T01:48:55.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:649: TEST_divergent_3: ceph osd pool set test size 3 2026-02-21T01:48:56.003 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 size to 3 2026-02-21T01:48:56.013 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:650: TEST_divergent_3: ceph osd pool set test min_size 2 2026-02-21T01:48:56.307 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 min_size to 2 2026-02-21T01:48:56.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:653: TEST_divergent_3: ceph osd pool set test pg_autoscale_mode on 2026-02-21T01:48:56.611 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 pg_autoscale_mode to on 2026-02-21T01:48:56.621 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:655: TEST_divergent_3: divergent=-1 2026-02-21T01:48:56.621 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:656: TEST_divergent_3: date +%s 2026-02-21T01:48:56.622 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:656: TEST_divergent_3: start_time=1771638536 2026-02-21T01:48:56.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: max_duration=300 2026-02-21T01:48:56.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: '[' -1 -le -1 ']' 2026-02-21T01:48:56.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:661: TEST_divergent_3: flush_pg_stats 2026-02-21T01:48:56.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:48:56.623 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:48:56.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:48:56.837 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:48:56.837 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:48:56.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:48:56.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:48:56.838 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:48:56.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-21T01:48:56.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-21T01:48:56.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-21T01:48:56.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:48:56.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:48:57.003 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672964 2026-02-21T01:48:57.003 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672964 2026-02-21T01:48:57.004 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-42949672964' 2026-02-21T01:48:57.004 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:48:57.004 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:48:57.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509443 2026-02-21T01:48:57.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509443 2026-02-21T01:48:57.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-42949672964 2-64424509443' 2026-02-21T01:48:57.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:48:57.087 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-21T01:48:57.087 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:48:57.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:48:57.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-21T01:48:57.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:48:57.090 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-21T01:48:57.090 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836485 2026-02-21T01:48:57.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-21T01:48:57.091 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:48:57.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836483 -lt 21474836485 2026-02-21T01:48:57.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:48:58.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:48:58.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:48:58.527 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-21T01:48:58.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:48:58.528 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672964 2026-02-21T01:48:58.528 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:48:58.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:48:58.530 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672964 2026-02-21T01:48:58.530 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:48:58.532 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672964 2026-02-21T01:48:58.532 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672964' 2026-02-21T01:48:58.532 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 42949672964 2026-02-21T01:48:58.532 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:48:58.758 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672964 -lt 42949672964 2026-02-21T01:48:58.758 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:48:58.758 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509443 2026-02-21T01:48:58.759 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:48:58.759 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:48:58.760 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509443 2026-02-21T01:48:58.760 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:48:58.761 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509443 2026-02-21T01:48:58.762 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 64424509443 2026-02-21T01:48:58.762 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509443' 2026-02-21T01:48:58.763 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:48:58.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509443 -lt 64424509443 2026-02-21T01:48:58.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:662: TEST_divergent_3: wait_for_clean 2026-02-21T01:48:58.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:48:58.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:48:58.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:48:58.972 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:48:58.973 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:48:58.973 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:48:58.973 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:48:58.973 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:48:58.973 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:48:59.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:48:59.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:48:59.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:48:59.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:48:59.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:48:59.074 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:48:59.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:48:59.300 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:48:59.300 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:48:59.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:48:59.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:48:59.300 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:48:59.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-21T01:48:59.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-21T01:48:59.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-21T01:48:59.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:48:59.380 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:48:59.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672967 2026-02-21T01:48:59.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672967 2026-02-21T01:48:59.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672967' 2026-02-21T01:48:59.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:48:59.463 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:48:59.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509446 2026-02-21T01:48:59.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509446 2026-02-21T01:48:59.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-42949672967 2-64424509446' 2026-02-21T01:48:59.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:48:59.547 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-21T01:48:59.547 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:48:59.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:48:59.548 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-21T01:48:59.549 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:48:59.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-21T01:48:59.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-21T01:48:59.551 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836487 2026-02-21T01:48:59.551 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:48:59.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-21T01:48:59.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:49:00.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:49:00.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:00.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-21T01:49:00.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:00.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672967 2026-02-21T01:49:00.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:00.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:49:00.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672967 2026-02-21T01:49:00.990 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:00.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672967 2026-02-21T01:49:00.991 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 42949672967 2026-02-21T01:49:00.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672967' 2026-02-21T01:49:00.992 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:49:01.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672967 -lt 42949672967 2026-02-21T01:49:01.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:01.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509446 2026-02-21T01:49:01.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:01.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:49:01.214 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509446 2026-02-21T01:49:01.214 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:01.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509446 2026-02-21T01:49:01.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509446' 2026-02-21T01:49:01.216 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 64424509446 2026-02-21T01:49:01.216 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:49:01.436 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509446 -lt 64424509446 2026-02-21T01:49:01.437 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:49:01.437 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:01.437 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:01.724 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:01.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:49:01.933 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:01.934 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:01.934 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:02.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:49:02.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:49:02.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:49:02.217 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: ceph pg dump pgs --format=json 2026-02-21T01:49:02.217 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: jq '.pg_stats[0].up_primary' 2026-02-21T01:49:02.411 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:49:02.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: divergent=1 2026-02-21T01:49:02.421 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:666: TEST_divergent_3: echo 'primary and soon to be divergent is 1' 2026-02-21T01:49:02.421 INFO:tasks.workunit.client.0.trial009.stdout:primary and soon to be divergent is 1 2026-02-21T01:49:02.421 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:49:02.628 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:49:02.628 INFO:tasks.workunit.client.0.trial009.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T01:48:55.125525+0000 0'0 30:22 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:16:24.381808+0000 0 0 2026-02-21T01:49:02.628 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:49:02.628 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:49:02.629 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:49:02.637 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: date +%s 2026-02-21T01:49:02.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: current_time=1771638542 2026-02-21T01:49:02.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:670: TEST_divergent_3: expr 1771638542 - 1771638536 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:670: TEST_divergent_3: elapsed_time=6 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:671: TEST_divergent_3: '[' 6 -gt 300 ']' 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: '[' 1 -le -1 ']' 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:677: TEST_divergent_3: local non_divergent= 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: for i in $osds 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:680: TEST_divergent_3: '[' 0 = 1 ']' 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:683: TEST_divergent_3: non_divergent=' 0' 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: for i in $osds 2026-02-21T01:49:02.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:680: TEST_divergent_3: '[' 1 = 1 ']' 2026-02-21T01:49:02.642 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: continue 2026-02-21T01:49:02.642 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: for i in $osds 2026-02-21T01:49:02.642 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:680: TEST_divergent_3: '[' 2 = 1 ']' 2026-02-21T01:49:02.642 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:683: TEST_divergent_3: non_divergent=' 0 2' 2026-02-21T01:49:02.642 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: echo 'writing initial objects' 2026-02-21T01:49:02.642 INFO:tasks.workunit.client.0.trial009.stdout:writing initial objects 2026-02-21T01:49:02.643 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: seq 1 100 2026-02-21T01:49:02.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_1 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_2 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.694 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.694 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_3 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.715 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.715 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_4 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_5 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_6 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.787 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.787 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_7 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.812 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_8 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_9 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.859 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.859 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_10 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_11 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_12 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.943 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.943 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_13 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_14 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:02.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:02.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_15 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_16 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.055 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.055 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_17 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_18 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.111 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_19 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_20 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_21 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.195 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.195 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_22 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_23 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_24 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.272 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_25 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.299 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.299 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_26 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.326 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_27 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_28 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_29 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.410 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.411 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_30 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.438 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.438 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_31 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.466 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.467 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_32 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_33 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_34 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_35 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_36 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_37 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.633 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.633 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_38 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.661 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.661 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_39 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.688 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.688 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_40 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_41 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_42 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_43 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_44 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_45 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_46 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_47 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_48 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.925 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.925 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_49 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.945 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_50 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.973 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.973 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_51 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:03.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:03.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_52 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.025 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_53 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_54 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_55 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.103 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.103 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_56 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_57 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_58 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_59 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.196 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.196 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_60 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_61 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_62 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.274 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_63 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_64 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_65 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_66 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_67 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_68 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.421 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.422 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_69 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.442 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.442 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_70 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.469 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_71 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_72 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_73 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_74 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.574 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_75 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_76 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.630 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.630 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_77 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_78 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_79 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.715 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_80 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_81 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.759 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.759 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_82 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_83 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_84 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_85 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.868 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.868 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_86 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.895 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_87 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_88 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_89 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_90 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:04.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:04.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_91 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.026 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.026 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_92 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_93 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.079 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_94 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.107 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.107 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_95 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_96 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_97 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_98 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_99 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.247 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-21T01:49:05.247 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:690: TEST_divergent_3: rados -p test put existing_100 /tmp/tmp.o88sKAF9En 2026-02-21T01:49:05.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:49:05.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: wait_for_clean 2026-02-21T01:49:05.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:49:05.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:49:05.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:49:05.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:49:05.277 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:49:05.277 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:49:05.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:49:05.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:49:05.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:49:05.349 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:49:05.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:49:05.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:49:05.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:49:05.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:49:05.350 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:49:05.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:49:05.568 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:49:05.568 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:49:05.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:49:05.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:05.568 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:49:05.652 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836491 2026-02-21T01:49:05.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836491 2026-02-21T01:49:05.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491' 2026-02-21T01:49:05.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:05.653 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:49:05.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=42949672970 2026-02-21T01:49:05.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 42949672970 2026-02-21T01:49:05.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970' 2026-02-21T01:49:05.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:05.732 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:49:05.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=64424509449 2026-02-21T01:49:05.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 64424509449 2026-02-21T01:49:05.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836491 1-42949672970 2-64424509449' 2026-02-21T01:49:05.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:05.815 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836491 2026-02-21T01:49:05.815 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:05.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:49:05.816 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836491 2026-02-21T01:49:05.816 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:05.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836491 2026-02-21T01:49:05.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-21T01:49:05.818 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836491 2026-02-21T01:49:05.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:06.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836491 2026-02-21T01:49:06.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:06.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-42949672970 2026-02-21T01:49:06.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:06.039 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:49:06.040 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-42949672970 2026-02-21T01:49:06.040 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:06.041 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=42949672970 2026-02-21T01:49:06.041 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 42949672970 2026-02-21T01:49:06.042 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 42949672970' 2026-02-21T01:49:06.042 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:49:06.262 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 42949672970 -lt 42949672970 2026-02-21T01:49:06.262 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:06.263 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-64424509449 2026-02-21T01:49:06.263 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:06.264 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:49:06.265 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-64424509449 2026-02-21T01:49:06.265 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:06.266 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=64424509449 2026-02-21T01:49:06.267 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 64424509449 2026-02-21T01:49:06.267 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 64424509449' 2026-02-21T01:49:06.267 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:49:06.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 64424509449 -lt 64424509449 2026-02-21T01:49:06.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:49:06.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:06.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:06.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:49:06.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:06.771 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:06.771 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:06.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:06.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:06.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:06.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:06.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:49:06.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:06.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:06.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:07.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:49:07.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:49:07.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:49:07.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:695: TEST_divergent_3: get_pg test existing_1 2026-02-21T01:49:07.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-21T01:49:07.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-21T01:49:07.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-21T01:49:07.277 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-21T01:49:07.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:695: TEST_divergent_3: local pgid=1.0 2026-02-21T01:49:07.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: echo 'blackholing osds 0 2' 2026-02-21T01:49:07.496 INFO:tasks.workunit.client.0.trial009.stdout:blackholing osds 0 2 2026-02-21T01:49:07.496 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:699: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:49:07.701 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:49:07.702 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-21T01:48:55.125525+0000 30'100 30:122 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:16:24.381808+0000 0 0 2026-02-21T01:49:07.702 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:49:07.702 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:49:07.702 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:49:07.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:07.713 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: get_asok_path osd.0 2026-02-21T01:49:07.713 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:49:07.713 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:49:07.714 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:49:07.714 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:49:07.714 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:49:07.714 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:49:07.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: CEPH_ARGS= 2026-02-21T01:49:07.715 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-21T01:49:07.768 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:49:07.768 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:49:07.768 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:49:07.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:07.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: get_asok_path osd.2 2026-02-21T01:49:07.774 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:49:07.775 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:49:07.775 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:49:07.775 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:49:07.775 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:49:07.775 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:49:07.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: CEPH_ARGS= 2026-02-21T01:49:07.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:702: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-21T01:49:07.827 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:49:07.828 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:49:07.828 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:49:07.833 INFO:tasks.workunit.client.0.trial009.stdout:create new divergent objects 2026-02-21T01:49:07.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:706: TEST_divergent_3: echo 'create new divergent objects' 2026-02-21T01:49:07.833 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: seq 1 2 2026-02-21T01:49:07.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:49:07.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:49:07.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:709: TEST_divergent_3: rados -p test create newobject_1 2026-02-21T01:49:07.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:712: TEST_divergent_3: echo 'writing divergent objects' 2026-02-21T01:49:07.835 INFO:tasks.workunit.client.0.trial009.stdout:writing divergent objects 2026-02-21T01:49:07.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:709: TEST_divergent_3: rados -p test create newobject_2 2026-02-21T01:49:07.836 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: seq 1 2 2026-02-21T01:49:07.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-21T01:49:07.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-21T01:49:07.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:715: TEST_divergent_3: rados -p test put existing_1 /tmp/tmp.clOFspByw9 2026-02-21T01:49:07.837 INFO:tasks.workunit.client.0.trial009.stdout:remove divergent objects 2026-02-21T01:49:07.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:718: TEST_divergent_3: echo 'remove divergent objects' 2026-02-21T01:49:07.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:715: TEST_divergent_3: rados -p test put existing_2 /tmp/tmp.clOFspByw9 2026-02-21T01:49:07.838 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: seq 1 2 2026-02-21T01:49:07.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-21T01:49:07.839 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: expr 1 + 2 2026-02-21T01:49:07.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: rmi=3 2026-02-21T01:49:07.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-21T01:49:07.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: rados -p test rm existing_3 2026-02-21T01:49:07.841 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: expr 2 + 2 2026-02-21T01:49:07.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:721: TEST_divergent_3: rmi=4 2026-02-21T01:49:07.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:724: TEST_divergent_3: sleep 10 2026-02-21T01:49:07.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: rados -p test rm existing_4 2026-02-21T01:49:17.844 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:725: TEST_divergent_3: killall -9 rados 2026-02-21T01:49:17.892 INFO:tasks.workunit.client.0.trial009.stderr:rados: no process found 2026-02-21T01:49:17.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: echo 'killing all the osds' 2026-02-21T01:49:17.893 INFO:tasks.workunit.client.0.trial009.stdout:killing all the osds 2026-02-21T01:49:17.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:49:18.102 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:49:18.103 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 active+clean 2026-02-21T01:48:55.125525+0000 30'106 30:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:16:24.381808+0000 0 0 2026-02-21T01:49:18.103 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:49:18.103 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:49:18.103 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:49:18.111 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:730: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd 2026-02-21T01:49:18.112 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:49:18.112 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:49:18.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:49:18.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:49:18.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:49:18.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:49:18.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: for i in $osds 2026-02-21T01:49:18.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: ceph osd down osd.0 2026-02-21T01:49:18.477 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:49:18.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: for i in $osds 2026-02-21T01:49:18.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: ceph osd down osd.1 2026-02-21T01:49:18.765 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:49:18.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: for i in $osds 2026-02-21T01:49:18.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: ceph osd down osd.2 2026-02-21T01:49:19.036 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:49:19.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:19.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:737: TEST_divergent_3: ceph osd out osd.0 2026-02-21T01:49:19.320 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already out. 2026-02-21T01:49:19.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:19.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:737: TEST_divergent_3: ceph osd out osd.2 2026-02-21T01:49:19.599 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already out. 2026-02-21T01:49:19.608 INFO:tasks.workunit.client.0.trial009.stdout:bringing up non_divergent 0 2 2026-02-21T01:49:19.608 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: echo 'bringing up non_divergent 0 2' 2026-02-21T01:49:19.608 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:742: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:49:19.819 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:49:19.820 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51152 0 0 16 90 16 stale+active+clean 2026-02-21T01:48:55.125525+0000 30'106 30:128 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:16:24.381808+0000 0 0 2026-02-21T01:49:19.820 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:49:19.820 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:49:19.820 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:49:19.831 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:19.831 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: activate_osd td/divergent-priors 0 2026-02-21T01:49:19.831 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:49:19.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:49:19.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:49:19.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:49:19.835 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:49:19.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:49:19.859 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:19.852+0000 7f3d153fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:49:19.861 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:19.852+0000 7f3d153fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:49:19.861 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:19.852+0000 7f3d153fca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:49:20.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:49:20.049 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:49:20.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:49:20.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:49:20.253 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:49:20.343 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:20.336+0000 7f3d153fca40 -1 Falling back to public interface 2026-02-21T01:49:21.028 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:21.020+0000 7f3d153fca40 -1 osd.0 30 log_to_monitors true 2026-02-21T01:49:21.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:49:21.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:49:21.255 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:49:21.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:49:21.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:49:21.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:49:21.472 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up out weight 0 up_from 37 up_thru 0 down_at 31 last_clean_interval [5,30) [v2:127.0.0.1:6800/4036132430,v1:127.0.0.1:6801/4036132430] [v2:127.0.0.1:6802/4036132430,v1:127.0.0.1:6803/4036132430] exists,up b6a9008d-1450-4e80-978d-4c4a7e607896 2026-02-21T01:49:21.475 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: activate_osd td/divergent-priors 2 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:49:21.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:49:21.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:49:21.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:49:21.479 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:49:21.479 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:49:21.479 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:49:21.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:49:21.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:49:21.480 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-21T01:49:21.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:49:21.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:49:21.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:49:21.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:49:21.498 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:21.492+0000 7ff9a0cbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:49:21.499 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:21.492+0000 7ff9a0cbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:49:21.500 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:21.492+0000 7ff9a0cbfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:49:21.697 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:49:21.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:49:21.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:49:21.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:49:21.910 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:49:21.935 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:21.928+0000 7ff9a0cbfa40 -1 Falling back to public interface 2026-02-21T01:49:22.615 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:49:22.608+0000 7ff9a0cbfa40 -1 osd.2 30 log_to_monitors true 2026-02-21T01:49:22.911 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:49:22.912 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:49:22.912 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:49:22.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:49:22.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:49:22.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:49:23.127 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up out weight 0 up_from 39 up_thru 0 down_at 33 last_clean_interval [15,30) [v2:127.0.0.1:6810/2349358508,v1:127.0.0.1:6811/2349358508] [v2:127.0.0.1:6812/2349358508,v1:127.0.0.1:6813/2349358508] exists,up 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 2026-02-21T01:49:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:49:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:49:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:49:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:749: TEST_divergent_3: ceph osd in osd.0 2026-02-21T01:49:23.373 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already in. 2026-02-21T01:49:23.385 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:49:23.385 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:749: TEST_divergent_3: ceph osd in osd.2 2026-02-21T01:49:23.646 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already in. 2026-02-21T01:49:23.656 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:752: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:49:23.656 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:752: TEST_divergent_3: wait_for_clean 2026-02-21T01:49:23.656 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:49:23.656 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:49:23.656 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:49:23.656 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:49:23.657 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:49:23.657 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:49:23.657 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:49:23.657 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:49:23.657 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:49:23.726 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:49:23.726 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:49:23.726 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:49:23.726 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:49:23.726 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:49:23.726 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:49:23.937 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:49:23.937 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:49:23.937 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:49:23.937 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:49:23.938 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:23.938 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:49:24.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789955 2026-02-21T01:49:24.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789955 2026-02-21T01:49:24.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-158913789955' 2026-02-21T01:49:24.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:24.019 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:49:24.080 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:49:24.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:49:24.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:49:24.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:49:24.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:24.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:49:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-21T01:49:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-21T01:49:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-158913789955 2-167503724547' 2026-02-21T01:49:24.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:24.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-158913789955 2026-02-21T01:49:24.164 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:24.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:49:24.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-158913789955 2026-02-21T01:49:24.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:24.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789955 2026-02-21T01:49:24.166 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 158913789955 2026-02-21T01:49:24.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 158913789955' 2026-02-21T01:49:24.167 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:24.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 158913789955 2026-02-21T01:49:24.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:49:25.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:49:25.379 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:25.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 158913789955 2026-02-21T01:49:25.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:49:26.602 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T01:49:26.603 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:26.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-21T01:49:26.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:26.815 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-167503724547 2026-02-21T01:49:26.816 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:26.817 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:49:26.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-167503724547 2026-02-21T01:49:26.818 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:26.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-21T01:49:26.819 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 167503724547 2026-02-21T01:49:26.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 167503724547' 2026-02-21T01:49:26.820 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:49:27.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-21T01:49:27.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:49:27.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:27.038 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:27.325 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:49:27.325 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:27.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:27.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:27.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:27.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:27.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:27.326 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:27.542 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:27.542 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:27.543 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:27.543 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:27.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:27.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:49:27.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:49:27.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:49:27.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:27.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:49:27.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:27.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:27.921 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:27.921 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:27.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:27.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:27.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:27.922 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:28.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:28.128 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:28.128 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:28.128 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:28.415 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:28.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:28.707 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:28.707 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:28.707 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:49:28.707 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:28.707 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:49:28.907 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:28.907 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:28.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:28.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:28.908 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:28.908 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:28.908 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:28.908 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:29.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:29.119 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:29.119 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:29.120 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:29.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:29.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:29.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:29.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:29.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:29.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:29.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:29.409 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:29.409 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:29.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:29.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:29.699 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:49:29.699 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:29.699 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:49:30.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:30.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:30.100 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:30.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:30.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:30.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:30.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:30.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:30.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:30.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:30.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:30.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:30.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:30.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:30.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:30.612 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:30.612 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:30.612 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:30.612 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:30.612 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:30.612 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:30.902 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:30.902 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:30.902 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:49:30.902 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:30.902 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:49:31.702 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:31.703 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:31.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:31.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:31.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:31.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:31.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:31.704 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:31.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:31.916 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:31.916 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:31.916 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:32.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:32.202 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:32.202 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:32.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:32.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:32.481 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:49:32.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:32.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:49:34.082 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:34.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:34.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:34.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:34.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:34.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:34.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:34.084 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:34.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:34.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:34.291 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:34.291 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:34.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:34.574 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:34.574 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:34.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:34.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:34.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:49:34.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:34.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:49:38.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:38.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:38.065 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:38.065 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:38.065 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:38.065 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:38.066 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:38.066 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:38.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:38.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:38.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:38.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:38.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:38.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:38.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:38.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:38.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:38.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:38.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:38.550 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:38.550 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:38.822 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:38.822 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:38.822 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:49:38.823 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:38.823 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:49:45.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:45.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:45.224 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:45.224 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:45.224 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:45.224 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:45.224 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:45.224 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:45.438 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:45.438 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:45.439 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:45.439 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:45.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:45.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:46.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:46.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:46.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:49:46.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:46.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:49:53.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:53.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:53.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:53.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:53.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:53.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:53.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:53.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:53.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:53.542 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:53.542 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:53.542 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:53.826 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:53.827 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:54.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:54.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:54.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:49:54.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:49:54.376 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:49:54.376 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "cb9d9bb6-752c-4338-8fb5-d5e783a42662", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:49:54.372686+0000", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:49:54.377 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.378 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: "count": 32 2026-02-21T01:49:54.379 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded+wait, acting [0,2]" 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.380 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.381 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.382 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.383 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.384 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.386 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.387 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:48:41.368931Z", 2026-02-21T01:49:54.388 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:48:41.368931Z", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:49:54.389 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.390 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.391 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:49:54.392 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 45, 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:48:41.448406+0000", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:49:45.977535+0000", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:49:22.766644+0000", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:49:23.587645+0000", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:49:54.393 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 9, 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:49:54.394 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:48:54.586388+0000", 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 1, 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool", 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:49:54.395 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 0, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "on", 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 32, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 32, 2026-02-21T01:49:54.396 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 32, 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 32, 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "44", 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:49:54.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "44", 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "selfmanaged", 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 0, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 0, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [], 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:49:54.398 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "", 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:49:54.399 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {}, 2026-02-21T01:49:54.400 INFO:tasks.workunit.client.0.trial009.stdout: "read_balance": { 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "score_type": "Fair distribution", 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "score_acting": 2.9900000095367432, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "score_stable": 2.9900000095367432, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_score": 0.67000001668930054, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_acting": 2, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_stable": 2, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity_weighted": 1, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity": 1, 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity_weighted": 1 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.401 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "b6a9008d-1450-4e80-978d-4c4a7e607896", 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 5, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 30, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 37, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 44, 2026-02-21T01:49:54.402 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 31, 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.403 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:49:54.404 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.405 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.406 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/4036132430", 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/4036132430", 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/4036132430", 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/4036132430", 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:49:54.407 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "130a8455-684c-4f56-9f61-eae6d72f68fc", 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:49:54.408 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 10, 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 22, 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 32, 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.409 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.410 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.411 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.412 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/1003373808", 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/1003373808", 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1003373808", 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1003373808", 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.413 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "11b9b532-e0f8-4e6d-8e3e-4b3e5577b821", 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 15, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 30, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 39, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 33, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.414 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:49:54.415 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.416 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:49:54.417 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/2349358508", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/2349358508", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2349358508", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2349358508", 2026-02-21T01:49:54.418 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:49:18.417834+0000", 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:48:44.947259+0000", 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:49:54.419 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:49:18.705192+0000", 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:48:48.627000+0000", 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:49:18.980036+0000", 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:49:54.420 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:48:52.742781+0000", 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:49:54.421 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.422 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/4036132430,v1:127.0.0.1:6803/4036132430]", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:49:54.423 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:48:42.427913Z", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:49:54.424 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/4036132430,v1:127.0.0.1:6801/4036132430]", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/4036132430,v1:127.0.0.1:6807/4036132430]", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/4036132430,v1:127.0.0.1:6805/4036132430]", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:49:54.425 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/1003373808,v1:127.0.0.1:6813/1003373808]", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:49:54.426 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:48:46.193797Z", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:49:54.427 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/1003373808,v1:127.0.0.1:6811/1003373808]", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1003373808,v1:127.0.0.1:6817/1003373808]", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1003373808,v1:127.0.0.1:6815/1003373808]", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:49:54.428 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/2349358508,v1:127.0.0.1:6813/2349358508]", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:49:54.429 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:48:50.242093Z", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/2349358508,v1:127.0.0.1:6811/2349358508]", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2349358508,v1:127.0.0.1:6817/2349358508]", 2026-02-21T01:49:54.430 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2349358508,v1:127.0.0.1:6815/2349358508]", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 0, 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [] 2026-02-21T01:49:54.431 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 37 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 40 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 45, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.432 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:49:54.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:49:54.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stderr:report 1846521048 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:49:54.435 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:49:54.436 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:49:54.437 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:49:54.438 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:49:54.439 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 0, 2026-02-21T01:49:54.440 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:48:41:448196+0000", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:49:54.441 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 32, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 32, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51200, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:49:54.442 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 100, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 100, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:49:54.443 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 512, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 512, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "up": 64, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 64, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 65, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:49:54.444 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83960, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2176, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 25, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81702, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488840, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036572160, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2228224, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 456546, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 26061, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83663411 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:49:54.445 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 65, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83960, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2176, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 25, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81702, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488840, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036572160, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2228224, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 456546, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 26061, 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83663411 2026-02-21T01:49:54.446 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 32, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51200, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:49:54.447 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 100, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 100, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:49:54.448 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1220608, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 153552, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 512, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 512, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "up": 64, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 64, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 158913789961 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 42949672972 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.449 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 167503724553 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded+wait", 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num": 1 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num": 31 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 32, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 32, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 32, 2026-02-21T01:49:54.450 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 3, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:49:49.855283+0000", 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "services": { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "osd": { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "daemons": { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "summary": "", 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "0": { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "2": { 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:49:54.451 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 37, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 219, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 312, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: expr 2 + 2 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: objname=existing_4 2026-02-21T01:49:54.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:756: TEST_divergent_3: echo 'writing non-divergent object existing_4' 2026-02-21T01:49:54.453 INFO:tasks.workunit.client.0.trial009.stdout:writing non-divergent object existing_4 2026-02-21T01:49:54.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:757: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988634+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988507+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988078+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988183+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986637+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988074+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986772+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989711+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989756+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989052+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989847+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989917+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986447+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.987928+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988134+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988911+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded+wait 2026-02-21T01:49:45.982589+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989107+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989675+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989387+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988480+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989257+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989036+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988026+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.588 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988382+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988203+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988818+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988716+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988546+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989871+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988488+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988995+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:49:54.589 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:49:54.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: rados -p test put existing_4 /tmp/tmp.clOFspByw9 2026-02-21T01:49:54.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:760: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:49:54.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:760: TEST_divergent_3: wait_for_clean 2026-02-21T01:49:54.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:49:54.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:49:54.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:49:54.627 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:49:54.628 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:49:54.628 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:49:54.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:49:54.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:49:54.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:49:54.700 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:49:54.700 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:49:54.701 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:49:54.701 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:49:54.701 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:49:54.701 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:49:54.924 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:49:54.924 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:49:54.925 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:49:54.925 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:49:54.925 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:54.925 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:49:55.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789963 2026-02-21T01:49:55.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789963 2026-02-21T01:49:55.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-158913789963' 2026-02-21T01:49:55.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:55.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:49:55.065 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:49:55.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:49:55.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:49:55.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:49:55.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:49:55.068 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:49:55.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724555 2026-02-21T01:49:55.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724555 2026-02-21T01:49:55.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-158913789963 2-167503724555' 2026-02-21T01:49:55.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-158913789963 2026-02-21T01:49:55.145 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:55.146 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:49:55.146 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-158913789963 2026-02-21T01:49:55.147 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:55.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789963 2026-02-21T01:49:55.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 158913789963' 2026-02-21T01:49:55.149 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 158913789963 2026-02-21T01:49:55.149 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:55.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789961 -lt 158913789963 2026-02-21T01:49:55.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:49:56.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:49:56.377 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:49:56.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789963 -lt 158913789963 2026-02-21T01:49:56.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:49:56.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-167503724555 2026-02-21T01:49:56.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:49:56.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:49:56.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-167503724555 2026-02-21T01:49:56.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:49:56.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724555 2026-02-21T01:49:56.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 167503724555' 2026-02-21T01:49:56.601 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 167503724555 2026-02-21T01:49:56.601 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:49:56.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724555 -lt 167503724555 2026-02-21T01:49:56.824 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:49:56.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:56.825 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:57.105 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 32 == 0 2026-02-21T01:49:57.105 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:57.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:57.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:57.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:57.106 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:57.106 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:57.106 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:57.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:57.308 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:57.309 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:57.309 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:57.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:49:57.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:49:57.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:49:57.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:49:57.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:57.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:49:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:57.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:57.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:57.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:57.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:57.682 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:57.682 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:57.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:57.891 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:57.892 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:57.892 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:58.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:49:58.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:58.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:58.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:58.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:58.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:58.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:58.168 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:58.168 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:58.442 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:58.442 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:58.442 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:49:58.443 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:58.443 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:49:58.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:58.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:58.644 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:58.644 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:58.645 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:58.645 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:58.645 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:58.645 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:49:58.852 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:49:58.853 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:49:58.853 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:49:58.853 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:49:59.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:49:59.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:49:59.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:49:59.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:49:59.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:49:59.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:49:59.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:49:59.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:49:59.810 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:49:59.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:49:59.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:49:59.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:49:59.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:49:59.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:49:59.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:50:00.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:50:00.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:50:00.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:50:00.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:50:00.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:50:00.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:50:00.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:50:00.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:50:00.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:50:00.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:50:00.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:50:00.320 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:50:00.320 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:50:00.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:50:00.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:50:00.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:50:00.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:50:00.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:50:01.412 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:50:01.412 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:50:01.413 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:50:01.413 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:50:01.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:50:01.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:50:01.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:50:01.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:50:01.631 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:50:01.631 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:50:01.632 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:50:01.632 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:50:01.919 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:50:01.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:50:02.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:50:02.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:50:02.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:50:02.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:50:02.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:50:03.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:50:03.801 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:50:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:50:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:50:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:50:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:50:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:50:03.802 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:50:04.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:50:04.017 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:50:04.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:50:04.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:50:04.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:50:04.307 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:50:04.307 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:50:04.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:50:04.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:50:04.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:50:04.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:50:04.597 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:50:07.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:50:07.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:50:07.798 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:50:07.798 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:50:07.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:50:07.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:50:07.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:50:07.799 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:50:08.013 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:50:08.014 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:50:08.014 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:50:08.015 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:50:08.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:50:08.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:50:08.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:50:08.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:50:08.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:50:08.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:50:08.293 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:50:08.293 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:50:08.293 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:50:08.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:50:08.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:50:08.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:50:08.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:50:08.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:50:14.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:50:14.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:50:14.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:50:14.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:50:14.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:50:14.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:50:14.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:50:14.980 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:50:15.192 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:50:15.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:50:15.194 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:50:15.194 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:50:15.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:50:15.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:50:15.478 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:50:15.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:50:15.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:50:15.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:50:15.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:50:15.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:50:23.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:50:23.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:50:23.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:50:23.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:50:23.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:50:23.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:50:23.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:50:23.072 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:50:23.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:50:23.291 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:50:23.292 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:50:23.292 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:50:23.586 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 32 2026-02-21T01:50:23.586 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:50:23.586 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:50:23.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:50:23.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:50:23.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:50:23.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:50:23.587 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:50:23.587 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:50:23.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:50:23.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:50:23.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:50:23.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:50:24.136 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:50:24.136 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "cb9d9bb6-752c-4338-8fb5-d5e783a42662", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:50:24.132553+0000", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:50:24.137 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.138 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: "count": 32 2026-02-21T01:50:24.139 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.140 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.141 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.142 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.143 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.144 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.145 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.145 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.145 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.145 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.146 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.146 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.146 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.146 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.146 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.146 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.147 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.148 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.148 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.148 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.148 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.148 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.148 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.149 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.149 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.149 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.149 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.149 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.149 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.150 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:48:41.368931Z", 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:48:41.368931Z", 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:50:24.151 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:50:24.152 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:50:24.153 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:50:24.154 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 45, 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:48:41.448406+0000", 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:49:45.977535+0000", 2026-02-21T01:50:24.155 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:49:22.766644+0000", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:49:23.587645+0000", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.156 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 9, 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:50:24.157 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:48:54.586388+0000", 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 1, 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool", 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:50:24.158 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 0, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "on", 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 32, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 32, 2026-02-21T01:50:24.159 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 32, 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 32, 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "44", 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:50:24.160 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "44", 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "selfmanaged", 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 0, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 0, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [], 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:50:24.161 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "", 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.162 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {}, 2026-02-21T01:50:24.163 INFO:tasks.workunit.client.0.trial009.stdout: "read_balance": { 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "score_type": "Fair distribution", 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "score_acting": 2.9900000095367432, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "score_stable": 2.9900000095367432, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_score": 0.67000001668930054, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_acting": 2, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "raw_score_stable": 2, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity_weighted": 1, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity": 1, 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "average_primary_affinity_weighted": 1 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.164 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "b6a9008d-1450-4e80-978d-4c4a7e607896", 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 5, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 30, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 37, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 44, 2026-02-21T01:50:24.165 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 31, 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.166 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.167 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.168 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:50:24.169 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 4036132430 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.170 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/4036132430", 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/4036132430", 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/4036132430", 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/4036132430", 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "130a8455-684c-4f56-9f61-eae6d72f68fc", 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:50:24.171 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 10, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 22, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 32, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.172 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.173 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.174 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.175 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1003373808 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/1003373808", 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/1003373808", 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1003373808", 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1003373808", 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.176 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "11b9b532-e0f8-4e6d-8e3e-4b3e5577b821", 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 15, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 30, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 39, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 33, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:50:24.177 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.178 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:50:24.179 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.180 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2349358508 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/2349358508", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/2349358508", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2349358508", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2349358508", 2026-02-21T01:50:24.181 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:49:18.417834+0000", 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:48:44.947259+0000", 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.182 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:49:18.705192+0000", 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:48:48.627000+0000", 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:49:18.980036+0000", 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:48:52.742781+0000", 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:50:24.183 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:50:24.184 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/4036132430,v1:127.0.0.1:6803/4036132430]", 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:50:24.185 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:50:24.186 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:48:42.427913Z", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/4036132430,v1:127.0.0.1:6801/4036132430]", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/4036132430,v1:127.0.0.1:6807/4036132430]", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/4036132430,v1:127.0.0.1:6805/4036132430]", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:50:24.187 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:50:24.188 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/1003373808,v1:127.0.0.1:6813/1003373808]", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:50:24.189 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:48:46.193797Z", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/1003373808,v1:127.0.0.1:6811/1003373808]", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1003373808,v1:127.0.0.1:6817/1003373808]", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1003373808,v1:127.0.0.1:6815/1003373808]", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:50:24.190 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/2349358508,v1:127.0.0.1:6813/2349358508]", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:50:24.191 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:48:50.242093Z", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:50:24.192 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/2349358508,v1:127.0.0.1:6811/2349358508]", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2349358508,v1:127.0.0.1:6817/2349358508]", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2349358508,v1:127.0.0.1:6815/2349358508]", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.193 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 0, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [] 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 37 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 40 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 45, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:50:24.194 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.195 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:50:24.196 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stderr:report 2858509188 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.197 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:50:24.198 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.199 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:50:24.200 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:50:24.201 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:50:24.202 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 0, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:48:41:448196+0000", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:50:24.203 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 32, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 32, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51688, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:50:24.204 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:50:24.205 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 513, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 513, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "up": 64, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 64, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 65, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:50:24.206 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83960, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2176, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 25, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81702, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488840, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036572160, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2228224, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 457522, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 26061, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83663411 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:50:24.207 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 65, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83960, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2176, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 25, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81702, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488840, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036572160, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2228224, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 457522, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 26061, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83663411 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:50:24.208 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 32, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 51688, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 100, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 300, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 100, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 100, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:50:24.209 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1220608, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 154528, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:50:24.210 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 513, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 513, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "up": 64, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 64, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 158913789969 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 42949672972 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 167503724561 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "num": 32 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:50:24.211 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 32, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 32, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 32, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 5, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:50:23.860412+0000", 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "services": {} 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 52, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:50:24.212 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 283, 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 391, 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: echo 'delay recovery' 2026-02-21T01:50:24.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:764: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:24.359 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:24.359 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988634+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.359 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988507+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988078+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988183+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986637+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988074+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986772+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989711+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989756+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989052+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989847+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989917+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986447+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.360 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.987928+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988134+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988911+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:55.694172+0000 30'100 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:50:46.140779+0000 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989107+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:49:45.989675+0000 45'101 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989387+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988480+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989257+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989036+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988026+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988382+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.361 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988203+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988818+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988716+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988546+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989871+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988488+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988995+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:24.362 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:24.368 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:50:24.368 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: get_asok_path osd.0 2026-02-21T01:50:24.368 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:50:24.368 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:50:24.369 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:50:24.369 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:50:24.369 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:50:24.369 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:50:24.370 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: CEPH_ARGS= 2026-02-21T01:50:24.370 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok set_recovery_delay 100000 2026-02-21T01:50:24.427 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:50:24.432 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: for i in $non_divergent 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: get_asok_path osd.2 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:50:24.434 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:50:24.435 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: CEPH_ARGS= 2026-02-21T01:50:24.435 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:767: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok set_recovery_delay 100000 2026-02-21T01:50:24.489 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:50:24.494 INFO:tasks.workunit.client.0.trial009.stdout:revive divergent 1 2026-02-21T01:50:24.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:771: TEST_divergent_3: echo 'revive divergent 1' 2026-02-21T01:50:24.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:772: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:24.704 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:24.704 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988634+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.704 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988507+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.704 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988078+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.704 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988183+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.704 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986637+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988074+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986772+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989711+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989756+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989052+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989847+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989917+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986447+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.987928+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988134+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.705 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988911+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:55.694172+0000 30'100 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:50:46.140779+0000 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989107+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:49:45.989675+0000 45'101 45:144 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989387+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988480+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989257+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989036+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988026+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988382+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988203+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.706 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988818+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988716+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988546+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989871+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988488+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988995+0000 30'100 45:143 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:24.707 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:24.713 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:773: TEST_divergent_3: ceph osd set noup 2026-02-21T01:50:25.043 INFO:tasks.workunit.client.0.trial009.stderr:noup is set 2026-02-21T01:50:25.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:774: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-21T01:50:25.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:50:25.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:50:25.058 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:50:25.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:50:25.060 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:50:25.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:50:25.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:50:25.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:50:25.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:50:25.061 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:50:25.062 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:50:25.063 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:50:25.063 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:50:25.064 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:50:25.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:50:25.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:50:25.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:50:25.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:50:25.081 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:25.071+0000 7f72d16e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:50:25.083 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:25.075+0000 7f72d16e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:50:25.084 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:25.075+0000 7f72d16e6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:50:25.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-21T01:50:25.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:775: TEST_divergent_3: sleep 5 2026-02-21T01:50:25.471 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:25.463+0000 7f72d16e6a40 -1 Falling back to public interface 2026-02-21T01:50:26.148 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:26.139+0000 7f72d16e6a40 -1 osd.1 30 log_to_monitors true 2026-02-21T01:50:30.279 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery divergent 2026-02-21T01:50:30.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: echo 'delay recovery divergent' 2026-02-21T01:50:30.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:778: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:30.476 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:30.476 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988634+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988507+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988078+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988183+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986637+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:02.146258+0000 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988074+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:06.146041+0000 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986772+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:05.122247+0000 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989711+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989756+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989052+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989847+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989917+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.477 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986447+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:50:58.074567+0000 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.987928+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988134+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988911+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:55.694172+0000 30'100 46:146 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:50:46.140779+0000 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989107+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:49:45.989675+0000 45'101 46:146 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989387+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988480+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989257+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989036+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988026+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:10.228955+0000 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988382+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.478 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988203+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988818+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988716+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988546+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989871+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988488+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988995+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:30.479 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:30.485 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: get_asok_path osd.1 2026-02-21T01:50:30.485 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:50:30.485 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:50:30.486 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:50:30.486 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:50:30.486 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:50:30.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:50:30.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: CEPH_ARGS= 2026-02-21T01:50:30.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok set_recovery_delay 100000 2026-02-21T01:50:30.542 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:50:30.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:781: TEST_divergent_3: ceph osd unset noup 2026-02-21T01:50:30.866 INFO:tasks.workunit.client.0.trial009.stderr:noup is unset 2026-02-21T01:50:30.879 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:783: TEST_divergent_3: wait_for_osd up 0 2026-02-21T01:50:30.879 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:50:30.879 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:50:30.879 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:50:30.879 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:50:30.879 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:50:30.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:50:30.880 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:50:30.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:50:30.881 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:50:31.096 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 37 up_thru 44 down_at 31 last_clean_interval [5,30) [v2:127.0.0.1:6800/4036132430,v1:127.0.0.1:6801/4036132430] [v2:127.0.0.1:6802/4036132430,v1:127.0.0.1:6803/4036132430] exists,up b6a9008d-1450-4e80-978d-4c4a7e607896 2026-02-21T01:50:31.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:50:31.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:784: TEST_divergent_3: wait_for_osd up 1 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:50:31.099 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:50:31.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:50:31.100 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:50:31.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:50:31.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:50:31.311 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/19377914,v1:127.0.0.1:6819/19377914] [v2:127.0.0.1:6820/19377914,v1:127.0.0.1:6821/19377914] exists,up 130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:50:31.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:50:31.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:50:31.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:50:31.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:785: TEST_divergent_3: wait_for_osd up 2 2026-02-21T01:50:31.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:50:31.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:50:31.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:50:31.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:50:31.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:50:31.313 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:50:31.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:50:31.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:50:31.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:50:31.516 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 39 up_thru 0 down_at 33 last_clean_interval [15,30) [v2:127.0.0.1:6810/2349358508,v1:127.0.0.1:6811/2349358508] [v2:127.0.0.1:6812/2349358508,v1:127.0.0.1:6813/2349358508] exists,up 11b9b532-e0f8-4e6d-8e3e-4b3e5577b821 2026-02-21T01:50:31.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:50:31.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:50:31.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:50:31.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:787: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:31.721 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:31.721 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988634+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.721 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988507+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.721 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988078+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.721 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988183+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.722 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986637+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:02.146258+0000 0 0 2026-02-21T01:50:31.722 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988074+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:06.146041+0000 0 0 2026-02-21T01:50:31.722 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986772+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:05.122247+0000 0 0 2026-02-21T01:50:31.722 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989711+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.722 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989756+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.722 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989052+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989847+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989917+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.986447+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:50:58.074567+0000 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.987928+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 4 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988134+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988911+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.723 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:55.694172+0000 30'100 46:146 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:50:46.140779+0000 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989107+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 3 0 0 2088 0 0 17 84 17 active+undersized+degraded 2026-02-21T01:49:45.989675+0000 45'101 46:146 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989387+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988480+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989257+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989036+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988026+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 user requested scrub scheduled @ 2026-02-21T01:51:10.228955+0000 0 0 2026-02-21T01:50:31.724 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988382+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988203+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988818+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988716+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988546+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.989871+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988488+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.725 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 3 0 0 1600 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:49:45.988995+0000 30'100 46:145 [0,2] 0 [0,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.726 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:31.726 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:31.726 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:31.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:788: TEST_divergent_3: echo 'wait for peering' 2026-02-21T01:50:31.731 INFO:tasks.workunit.client.0.trial009.stdout:wait for peering 2026-02-21T01:50:31.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:789: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:31.936 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:31.936 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230938+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.936 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.230596+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.936 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229691+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.936 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.224034+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.228097+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 1 1 0 0 1600 0 0 11 89 11 active+recovering+degraded 2026-02-21T01:50:31.230589+0000 30'100 51:140 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.224182+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230111+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.227097+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.225915+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.226978+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.226402+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.225865+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-21T01:50:31.226967+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 0 0 0 1600 0 0 15 85 15 active+clean 2026-02-21T01:50:31.228705+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.937 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229916+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.222337+0000 30'100 51:141 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 0 0 0 1600 0 0 12 88 12 active+clean 2026-02-21T01:50:31.222817+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 1 1 0 0 2088 0 0 14 87 14 active+recovery_wait+degraded 2026-02-21T01:50:31.228048+0000 45'101 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229991+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.227105+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 1 1 0 0 1600 0 0 16 84 16 active+recovery_wait+degraded 2026-02-21T01:50:31.229680+0000 30'100 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229397+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.229610+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.229913+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.226988+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.938 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229837+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230712+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.225468+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229810+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229902+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 1 1 0 0 1600 0 0 10 90 10 active+recovery_wait+degraded 2026-02-21T01:50:31.228355+0000 30'100 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:31.939 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:31.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:790: TEST_divergent_3: rados -p test put foo /tmp/tmp.o88sKAF9En 2026-02-21T01:50:31.968 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:791: TEST_divergent_3: rm -f /tmp/tmp.o88sKAF9En 2026-02-21T01:50:31.969 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: rm -f /tmp/tmp.clOFspByw9 2026-02-21T01:50:31.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:796: TEST_divergent_3: echo 'killing divergent 1' 2026-02-21T01:50:31.970 INFO:tasks.workunit.client.0.trial009.stdout:killing divergent 1 2026-02-21T01:50:31.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:32.176 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230938+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.230596+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229691+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.224034+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.228097+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 1 1 0 0 1600 0 0 11 89 11 active+recovering+degraded 2026-02-21T01:50:31.230589+0000 30'100 51:140 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.224182+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230111+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.227097+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.177 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.225915+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.226978+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.226402+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.225865+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-21T01:50:31.226967+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 0 0 0 1600 0 0 15 85 15 active+clean 2026-02-21T01:50:31.228705+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229916+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.222337+0000 30'100 51:141 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 0 0 0 1600 0 0 12 88 12 active+clean 2026-02-21T01:50:31.222817+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 1 1 0 0 2088 0 0 14 87 14 active+recovery_wait+degraded 2026-02-21T01:50:31.228048+0000 45'101 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229991+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.178 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.227105+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 1 1 0 0 1600 0 0 16 84 16 active+recovery_wait+degraded 2026-02-21T01:50:31.229680+0000 30'100 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229397+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.229610+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.229913+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.226988+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229837+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230712+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.225468+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229810+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229902+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:32.179 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 1 1 0 0 1600 0 0 10 90 10 active+recovery_wait+degraded 2026-02-21T01:50:31.228355+0000 30'100 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:32.180 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:32.180 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:32.180 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:32.185 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:798: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd.1 2026-02-21T01:50:32.186 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:50:32.186 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:50:32.186 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:50:32.186 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:50:32.186 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:50:32.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:801: TEST_divergent_3: expfile=td/divergent-priors/exp.20522.out 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:802: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20522.out 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-21T01:50:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20522.out 2026-02-21T01:50:32.454 INFO:tasks.workunit.client.0.trial009.stderr:Exporting 1.0 info 1.0( v 30'100 (30'90,30'100] local-lis/les=50/51 n=3 ec=22/22 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-21T01:50:32.455 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-21T01:50:32.455 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-21T01:50:32.455 INFO:tasks.workunit.client.0.trial009.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-21T01:50:32.455 INFO:tasks.workunit.client.0.trial009.stderr:Export successful 2026-02-21T01:50:32.456 INFO:tasks.workunit.client.0.trial009.stdout: marking collection for removal 2026-02-21T01:50:32.456 INFO:tasks.workunit.client.0.trial009.stdout:setting '_remove' omap key 2026-02-21T01:50:32.456 INFO:tasks.workunit.client.0.trial009.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-21T01:50:32.456 INFO:tasks.workunit.client.0.trial009.stderr:Remove successful 2026-02-21T01:50:32.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:803: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.20522.out 2026-02-21T01:50:32.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:50:32.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:50:32.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T01:50:32.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:50:32.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-21T01:50:32.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.20522.out 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:get_pg_num_history pg_num_history pg_num_history(e51 pg_nums {1={22=1,44=32}} deleted_pools ) 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:Importing pgid 1.0 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:snapset 0={} 2026-02-21T01:50:32.722 INFO:tasks.workunit.client.0.trial009.stdout:write_pg epoch 51 info 1.0( v 30'100 (30'90,30'100] local-lis/les=50/51 n=3 ec=22/22 lis/c=50/50 les/c/f=51/51/0 sis=50) 2026-02-21T01:50:32.723 INFO:tasks.workunit.client.0.trial009.stdout:Import successful 2026-02-21T01:50:32.819 INFO:tasks.workunit.client.0.trial009.stdout:reviving divergent 1 2026-02-21T01:50:32.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:805: TEST_divergent_3: echo 'reviving divergent 1' 2026-02-21T01:50:32.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:33.031 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:33.031 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230938+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.031 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.230596+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.031 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229691+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.031 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.224034+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.031 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.228097+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.9 3 1 1 0 0 1600 0 0 11 89 11 active+recovering+degraded 2026-02-21T01:50:31.230589+0000 30'100 51:140 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.224182+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230111+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.6 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.227097+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.225915+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.4 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.226978+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.226402+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.2 4 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.225865+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-21T01:50:31.226967+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.1 4 0 0 0 0 1600 0 0 15 85 15 active+clean 2026-02-21T01:50:31.228705+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.032 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229916+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.222337+0000 30'100 51:141 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 0 0 0 1600 0 0 12 88 12 active+clean 2026-02-21T01:50:31.222817+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 1 1 0 0 2088 0 0 14 87 14 active+recovery_wait+degraded 2026-02-21T01:50:31.228048+0000 45'101 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229991+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.227105+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 1 1 0 0 1600 0 0 16 84 16 active+recovery_wait+degraded 2026-02-21T01:50:31.229680+0000 30'100 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.12 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229397+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.229610+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.229913+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.226988+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.033 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229837+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.230712+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:31.225468+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout:1.19 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229810+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:31.229902+0000 30'100 51:142 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 1 1 0 0 1600 0 0 10 90 10 active+recovery_wait+degraded 2026-02-21T01:50:31.228355+0000 30'100 51:139 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:33.034 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:50:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:50:33.044 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:50:33.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:50:33.046 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:50:33.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:50:33.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:50:33.047 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:50:33.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:50:33.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:50:33.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:50:33.048 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:50:33.071 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:33.063+0000 7f34c1ee6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:50:33.072 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:33.063+0000 7f34c1ee6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:50:33.073 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:33.063+0000 7f34c1ee6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:50:33.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:50:33.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:50:33.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:50:33.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:50:33.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:50:33.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:50:33.256 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:50:33.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:50:33.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:50:33.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:50:33.475 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/19377914,v1:127.0.0.1:6819/19377914] [v2:127.0.0.1:6820/19377914,v1:127.0.0.1:6821/19377914] exists,up 130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:50:33.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:50:33.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:50:33.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:50:33.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:808: TEST_divergent_3: wait_for_osd up 1 2026-02-21T01:50:33.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:50:33.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:50:33.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:50:33.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:50:33.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:50:33.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:50:33.477 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:50:33.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:50:33.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:50:33.507 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:33.499+0000 7f34c1ee6a40 -1 Falling back to public interface 2026-02-21T01:50:33.686 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 50 up_thru 50 down_at 32 last_clean_interval [10,30) [v2:127.0.0.1:6818/19377914,v1:127.0.0.1:6819/19377914] [v2:127.0.0.1:6820/19377914,v1:127.0.0.1:6821/19377914] exists,up 130a8455-684c-4f56-9f61-eae6d72f68fc 2026-02-21T01:50:33.686 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:50:33.686 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:50:33.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:50:33.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: sleep 20 2026-02-21T01:50:34.149 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:50:34.139+0000 7f34c1ee6a40 -1 osd.1 51 log_to_monitors true 2026-02-21T01:50:53.688 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:811: TEST_divergent_3: get_asok_path osd.1 2026-02-21T01:50:53.688 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:50:53.689 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:50:53.689 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:50:53.690 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:50:53.690 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:50:53.690 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:50:53.690 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:811: TEST_divergent_3: CEPH_ARGS= 2026-02-21T01:50:53.690 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:811: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok dump_ops_in_flight 2026-02-21T01:50:53.745 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:50:53.745 INFO:tasks.workunit.client.0.trial009.stdout: "ops": [], 2026-02-21T01:50:53.745 INFO:tasks.workunit.client.0.trial009.stdout: "num_ops": 0 2026-02-21T01:50:53.745 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:50:53.751 INFO:tasks.workunit.client.0.trial009.stdout:allowing recovery 2026-02-21T01:50:53.751 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:813: TEST_divergent_3: echo 'allowing recovery' 2026-02-21T01:50:53.751 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:53.967 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:53.967 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022641+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:42.023242+0000 30'100 61:168 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-21T01:50:41.862867+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:41.862829+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:56:18.984594+0000 4 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.023103+0000 30'100 63:171 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:42.022655+0000 30'100 65:179 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.9 5 0 0 0 0 2560 0 0 16 84 16 active+clean 2026-02-21T01:50:49.856376+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:49.856327+0000 0'0 2026-02-21T01:48:54.610005+0000 0 1 periodic scrub scheduled @ 2026-02-22T12:34:33.643206+0000 5 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-21T01:50:43.098370+0000 30'100 63:182 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-21T01:50:43.098308+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:43:43.141473+0000 3 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022978+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.6 4 0 0 0 0 2112 0 0 17 84 17 active+clean 2026-02-21T01:50:42.022770+0000 51'101 61:159 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:40.020949+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.4 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-21T01:50:35.146547+0000 30'100 63:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-21T01:50:35.146503+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:22:54.790522+0000 2 0 2026-02-21T01:50:53.968 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.022973+0000 30'100 63:179 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-21T01:50:37.145982+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.2 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-21T01:50:43.260802+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-21T01:50:43.260751+0000 0'0 2026-02-21T01:48:54.610005+0000 0 1 periodic scrub scheduled @ 2026-02-22T08:53:35.693530+0000 1 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-21T01:50:34.550527+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.1 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-21T01:50:40.860168+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:40.860122+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:23:50.907131+0000 2 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:34.551198+0000 30'100 63:173 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-21T01:50:47.898613+0000 30'100 63:182 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-21T01:50:47.898569+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T12:42:33.531256+0000 3 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:40.020951+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 0 0 0 2088 0 0 17 84 17 active+clean 2026-02-21T01:50:42.022803+0000 45'101 61:169 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022200+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:34.546903+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-21T01:50:45.876144+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:45.876088+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:17:46.119470+0000 3 0 2026-02-21T01:50:53.969 INFO:tasks.workunit.client.0.trial009.stdout:1.12 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-21T01:50:49.199127+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-21T01:50:49.199080+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T12:47:10.115307+0000 1 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.023368+0000 30'100 63:171 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:34.549870+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:40.020715+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:42.023053+0000 30'100 61:168 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022377+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:34.546571+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.19 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-21T01:50:42.841544+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:42.841505+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:42:55.751934+0000 2 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-21T01:50:45.285698+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-21T01:50:45.285646+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:53:28.228111+0000 4 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.023558+0000 30'100 63:172 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:53.970 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:53.971 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:53.977 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: for i in $osds 2026-02-21T01:50:53.977 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-21T01:50:54.049 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:50:54.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: for i in $osds 2026-02-21T01:50:54.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-21T01:50:54.144 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:50:54.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:816: TEST_divergent_3: for i in $osds 2026-02-21T01:50:54.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-21T01:50:54.234 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:50:54.242 INFO:tasks.workunit.client.0.trial009.stdout:reading divergent objects 2026-02-21T01:50:54.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: echo 'reading divergent objects' 2026-02-21T01:50:54.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:822: TEST_divergent_3: ceph pg dump pgs 2026-02-21T01:50:54.463 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:50:54.463 INFO:tasks.workunit.client.0.trial009.stdout:1.1f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022641+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.463 INFO:tasks.workunit.client.0.trial009.stdout:1.1e 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:42.023242+0000 30'100 61:168 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.1d 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-21T01:50:41.862867+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:41.862829+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:56:18.984594+0000 4 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.023103+0000 30'100 63:171 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.a 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:42.022655+0000 30'100 65:179 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.9 5 0 0 0 0 2560 0 0 16 84 16 active+clean 2026-02-21T01:50:49.856376+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:49.856327+0000 0'0 2026-02-21T01:48:54.610005+0000 0 1 periodic scrub scheduled @ 2026-02-22T12:34:33.643206+0000 5 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.8 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-21T01:50:43.098370+0000 30'100 63:182 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-21T01:50:43.098308+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:43:43.141473+0000 3 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.7 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022978+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.6 4 0 0 0 0 2112 0 0 17 84 17 active+clean 2026-02-21T01:50:42.022770+0000 51'101 61:159 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.5 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:40.020949+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.4 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-21T01:50:35.146547+0000 30'100 63:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-21T01:50:35.146503+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:22:54.790522+0000 2 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.3 4 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.022973+0000 30'100 63:179 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-21T01:50:37.145982+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.464 INFO:tasks.workunit.client.0.trial009.stdout:1.2 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-21T01:50:43.260802+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-21T01:50:43.260751+0000 0'0 2026-02-21T01:48:54.610005+0000 0 1 periodic scrub scheduled @ 2026-02-22T08:53:35.693530+0000 1 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.c 3 0 0 0 0 1600 0 0 14 86 14 active+clean 2026-02-21T01:50:34.550527+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.1 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-21T01:50:40.860168+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:40.860122+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:23:50.907131+0000 2 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.1c 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:34.551198+0000 30'100 63:173 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.0 3 0 0 0 0 1536 0 0 10 90 10 active+clean 2026-02-21T01:50:47.898613+0000 30'100 63:182 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-21T01:50:47.898569+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T12:42:33.531256+0000 3 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.d 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:40.020951+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.e 3 0 0 0 0 2088 0 0 17 84 17 active+clean 2026-02-21T01:50:42.022803+0000 45'101 61:169 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.f 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022200+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.10 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:34.546903+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.11 3 0 0 0 0 1536 0 0 16 84 16 active+clean 2026-02-21T01:50:45.876144+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:45.876088+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:17:46.119470+0000 3 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.12 1 0 0 0 0 512 0 0 16 84 16 active+clean 2026-02-21T01:50:49.199127+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-21T01:50:49.199080+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T12:47:10.115307+0000 1 0 2026-02-21T01:50:54.465 INFO:tasks.workunit.client.0.trial009.stdout:1.13 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.023368+0000 30'100 63:171 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.14 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:34.549870+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.15 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:40.020715+0000 30'100 66:157 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.16 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:42.023053+0000 30'100 61:168 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.17 3 0 0 0 0 1600 0 0 16 84 16 active+clean 2026-02-21T01:50:44.022377+0000 30'100 63:169 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.18 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:34.546571+0000 30'100 63:174 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.19 2 0 0 0 0 1024 0 0 16 84 16 active+clean 2026-02-21T01:50:42.841544+0000 30'100 66:165 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-21T01:50:42.841505+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T07:42:55.751934+0000 2 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.1a 4 0 0 0 0 2048 0 0 16 84 16 active+clean 2026-02-21T01:50:45.285698+0000 30'100 65:187 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-21T01:50:45.285646+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 periodic scrub scheduled @ 2026-02-22T03:53:28.228111+0000 4 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:1.1b 3 0 0 0 0 1600 0 0 10 90 10 active+clean 2026-02-21T01:50:44.023558+0000 30'100 63:172 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-21T01:48:54.610005+0000 0'0 2026-02-21T01:48:54.610005+0000 0 0 queued for scrub 0 0 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:50:54.466 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:50:54.473 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: expr 2 + 2 2026-02-21T01:50:54.474 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: seq 1 4 2026-02-21T01:50:54.475 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:50:54.475 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_1 td/divergent-priors/existing 2026-02-21T01:50:54.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:50:54.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_2 td/divergent-priors/existing 2026-02-21T01:50:54.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:50:54.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_3 td/divergent-priors/existing 2026-02-21T01:50:54.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:50:54.553 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get existing_4 td/divergent-priors/existing 2026-02-21T01:50:54.582 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: seq 1 2 2026-02-21T01:50:54.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:50:54.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-21T01:50:54.603 INFO:tasks.workunit.client.0.trial009.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-21T01:50:54.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-21T01:50:54.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-21T01:50:54.631 INFO:tasks.workunit.client.0.trial009.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-21T01:50:54.633 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: rm -f td/divergent-priors/existing 2026-02-21T01:50:54.635 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:833: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-21T01:50:54.636 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:833: TEST_divergent_3: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:50:54.639 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4543.0:1 2026-02-21T01:49:07.853028+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 20 _merge_object_divergent_entries: keeping 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4543.0:1 2026-02-21T01:49:07.853028+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 30'2 first_divergent_update: 30'104 last_divergent_update: 30'104 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 30'106 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4543.0:1 2026-02-21T01:49:07.853028+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 30'2 <= info.log_tail 30'89 2026-02-21T01:50:54.640 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 30'106 (30'4) delete 1:70b3bf12:::existing_4:head by client.4549.0:1 2026-02-21T01:49:07.858817+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 20 _merge_object_divergent_entries: keeping 30'106 (30'4) delete 1:70b3bf12:::existing_4:head by client.4549.0:1 2026-02-21T01:49:07.858817+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 30'4 first_divergent_update: 30'106 last_divergent_update: 30'106 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: more recent entry found: 45'101 (30'4) modify 1:70b3bf12:::existing_4:head by client.4660.0:1 2026-02-21T01:49:54.620682+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false, already merged 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 30'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4539.0:1 2026-02-21T01:49:07.851312+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 20 _merge_object_divergent_entries: keeping 30'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4539.0:1 2026-02-21T01:49:07.851312+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 30'101 last_divergent_update: 30'101 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.215+0000 7f72abe5d640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4546.0:1 2026-02-21T01:49:07.855251+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:50:54.641 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 20 _merge_object_divergent_entries: keeping 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4546.0:1 2026-02-21T01:49:07.855251+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 30'3 first_divergent_update: 30'105 last_divergent_update: 30'105 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 30'106 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4546.0:1 2026-02-21T01:49:07.855251+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: false, new_object: false 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72ae662640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 30'3 <= info.log_tail 30'84 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 30'102 (30'1) modify 1:d858c035:::existing_1:head by client.4541.0:1 2026-02-21T01:49:07.852156+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:50:54.642 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 20 _merge_object_divergent_entries: keeping 30'102 (30'1) modify 1:d858c035:::existing_1:head by client.4541.0:1 2026-02-21T01:49:07.852156+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 30'1 first_divergent_update: 30'102 last_divergent_update: 30'102 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 30'106 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 30'102 (30'1) modify 1:d858c035:::existing_1:head by client.4541.0:1 2026-02-21T01:49:07.852156+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 30'1 <= info.log_tail 30'90 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 30'103 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4542.0:1 2026-02-21T01:49:07.852561+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:50:54.648 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 20 _merge_object_divergent_entries: keeping 30'103 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4542.0:1 2026-02-21T01:49:07.852561+0000 0 ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:50:54.649 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:50:54.649 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 30'103 last_divergent_update: 30'103 2026-02-21T01:50:54.649 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-21T01:50:54.649 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:50:31.219+0000 7f72b1668640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-21T01:50:54.649 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:835: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-21T01:50:54.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:835: TEST_divergent_3: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:50:54.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:841: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-21T01:50:54.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:841: TEST_divergent_3: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:50:54.656 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:846: TEST_divergent_3: echo success 2026-02-21T01:50:54.656 INFO:tasks.workunit.client.0.trial009.stdout:success 2026-02-21T01:50:54.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:848: TEST_divergent_3: rm td/divergent-priors/td/divergent-priors/exp.20522.out 2026-02-21T01:50:54.657 INFO:tasks.workunit.client.0.trial009.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.20522.out': No such file or directory 2026-02-21T01:50:54.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:849: TEST_divergent_3: delete_pool test 2026-02-21T01:50:54.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-21T01:50:54.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-21T01:50:54.879 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' does not exist 2026-02-21T01:50:54.889 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:850: TEST_divergent_3: kill_daemons td/divergent-priors 2026-02-21T01:50:54.889 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:50:54.890 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:50:54.890 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:50:54.890 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:50:54.890 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:51:00.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:51:00.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:51:00.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:51:00.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:51:00.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:51:00.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:51:00.213 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:51:00.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:51:00.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:51:00.215 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:51:00.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:51:00.216 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:51:00.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:51:00.217 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:51:00.218 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:51:00.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:51:00.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:51:00.239 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:51:00.240 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.240 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.240 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:51:00.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:51:00.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:51:00.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/divergent-priors 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/divergent-priors 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:51:00.243 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:51:00.244 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:51:00.244 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:51:00.244 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:51:00.244 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:51:00.245 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:51:00.245 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:51:00.246 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:51:00.246 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:51:00.247 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:51:00.247 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:51:00.247 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:51:00.248 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:51:00.248 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:51:00.248 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:51:00.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:51:00.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:51:00.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:51:00.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:51:00.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:51:00.253 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:51:00.253 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:51:00.253 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/divergent-priors 2026-02-21T01:51:00.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:51:00.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.254 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.20522 2026-02-21T01:51:00.255 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:51:00.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T01:51:00.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:51:00.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-21T01:51:00.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_ec td/divergent-priors 2026-02-21T01:51:00.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:227: TEST_divergent_ec: local dir=td/divergent-priors 2026-02-21T01:51:00.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:229: TEST_divergent_ec: file_with_random_data 2026-02-21T01:51:00.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=512 2026-02-21T01:51:00.258 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.S3GfTrhjDp bs=512 count=1 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr:512 bytes copied, 5.2467e-05 s, 9.8 MB/s 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:229: TEST_divergent_ec: local dummyfile=/tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:00.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: file_with_random_data 1000 2026-02-21T01:51:00.261 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2344: file_with_random_data: local size_bytes=1000 2026-02-21T01:51:00.261 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: mktemp 2026-02-21T01:51:00.262 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2345: file_with_random_data: local file=/tmp/tmp.3wgIKZgB5J 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2346: file_with_random_data: dd if=/dev/urandom of=/tmp/tmp.3wgIKZgB5J bs=1000 count=1 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records in 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr:1+0 records out 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr:1000 bytes (1.0 kB) copied, 8.1452e-05 s, 12.3 MB/s 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2347: file_with_random_data: printf %s /tmp/tmp.3wgIKZgB5J 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: local dummyfile2=/tmp/tmp.3wgIKZgB5J 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:233: TEST_divergent_ec: local num_osds=3 2026-02-21T01:51:00.263 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: expr 3 - 1 2026-02-21T01:51:00.264 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: seq 0 2 2026-02-21T01:51:00.264 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: local 'osds=0 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: run_mon td/divergent-priors a 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/divergent-priors 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/divergent-priors/a 2026-02-21T01:51:00.265 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:00.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:51:00.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:51:00.317 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:51:00.318 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.318 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.318 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:51:00.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:51:00.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get fsid 2026-02-21T01:51:00.372 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:51:00.372 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:51:00.372 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:51:00.372 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:51:00.373 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-mon.a.asok 2026-02-21T01:51:00.374 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:51:00.375 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.20522/ceph-mon.a.asok config get mon_host 2026-02-21T01:51:00.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:236: TEST_divergent_ec: run_mgr td/divergent-priors x 2026-02-21T01:51:00.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/divergent-priors 2026-02-21T01:51:00.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:51:00.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:51:00.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:51:00.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/divergent-priors/x 2026-02-21T01:51:00.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:51:00.547 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:51:00.547 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:00.548 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:00.548 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:00.548 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.548 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.548 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:00.549 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:51:00.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:239: TEST_divergent_ec: run_osd td/divergent-priors 0 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:51:00.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:00.582 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:51:00.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:51:00.584 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:51:00.584 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:51:00.584 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:51:00.584 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:51:00.584 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:51:00.584 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:51:00.585 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:51:00.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 bbfde735-2cd5-4c97-af04-f608bbd9a42f' 2026-02-21T01:51:00.585 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:51:00.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCED5lpBzc+IxAAzPNpZ0hWd9iQQJI1q8Ovng== 2026-02-21T01:51:00.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCED5lpBzc+IxAAzPNpZ0hWd9iQQJI1q8Ovng=="}' 2026-02-21T01:51:00.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new bbfde735-2cd5-4c97-af04-f608bbd9a42f -i td/divergent-priors/0/new.json 2026-02-21T01:51:00.710 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:51:00.718 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/0/new.json 2026-02-21T01:51:00.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCED5lpBzc+IxAAzPNpZ0hWd9iQQJI1q8Ovng== --osd-uuid bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:51:00.738 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:00.734+0000 7f4278f87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:00.740 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:00.734+0000 7f4278f87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:00.741 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:00.734+0000 7f4278f87a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:00.741 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:00.734+0000 7f4278f87a40 -1 bdev(0x5573f256ec00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:51:00.741 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:00.734+0000 7f4278f87a40 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-21T01:51:01.490 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-21T01:51:01.490 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:51:01.491 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:51:01.491 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:51:01.491 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:51:01.616 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:51:01.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:51:01.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:51:01.617 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:51:01.618 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:51:01.618 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:51:01.647 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:01.642+0000 7f0f578f5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:01.649 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:01.642+0000 7f0f578f5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:01.650 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:01.646+0000 7f0f578f5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:01.726 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:51:01.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:51:01.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:51:01.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:51:01.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:51:01.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:01.727 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:51:01.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:51:01.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:01.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:51:01.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:02.127 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:02.122+0000 7f0f578f5a40 -1 Falling back to public interface 2026-02-21T01:51:02.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:02.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:02.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:51:02.836 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:51:02.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:02.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:51:03.037 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:03.030+0000 7f0f578f5a40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:51:03.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:04.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:04.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:04.045 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:51:04.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:51:04.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:04.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:51:04.249 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2904256892,v1:127.0.0.1:6801/2904256892] [v2:127.0.0.1:6802/2904256892,v1:127.0.0.1:6803/2904256892] exists,up bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:51:04.254 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:239: TEST_divergent_ec: run_osd td/divergent-priors 1 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:51:04.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:51:04.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:51:04.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:51:04.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:51:04.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:51:04.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:51:04.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:04.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:51:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:51:04.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:51:04.259 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:51:04.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=24ff08c5-1b74-4350-ade4-0abd133531ca 2026-02-21T01:51:04.259 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 24ff08c5-1b74-4350-ade4-0abd133531ca 2026-02-21T01:51:04.260 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 24ff08c5-1b74-4350-ade4-0abd133531ca' 2026-02-21T01:51:04.260 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:51:04.269 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCID5lp+oLXDxAAKqKi8iCHPk3NrkEXpTCPKg== 2026-02-21T01:51:04.270 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCID5lp+oLXDxAAKqKi8iCHPk3NrkEXpTCPKg=="}' 2026-02-21T01:51:04.270 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 24ff08c5-1b74-4350-ade4-0abd133531ca -i td/divergent-priors/1/new.json 2026-02-21T01:51:04.494 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:51:04.505 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/1/new.json 2026-02-21T01:51:04.506 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCID5lp+oLXDxAAKqKi8iCHPk3NrkEXpTCPKg== --osd-uuid 24ff08c5-1b74-4350-ade4-0abd133531ca 2026-02-21T01:51:04.532 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:04.526+0000 7f9238539a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:04.534 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:04.526+0000 7f9238539a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:04.535 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:04.530+0000 7f9238539a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:04.535 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:04.530+0000 7f9238539a40 -1 bdev(0x5567b4530c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:51:04.535 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:04.530+0000 7f9238539a40 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-21T01:51:05.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-21T01:51:05.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:51:05.218 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:51:05.219 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:51:05.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:51:05.510 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:51:05.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:51:05.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:51:05.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:51:05.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:51:05.511 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:51:05.529 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:05.522+0000 7fb31e7a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:05.531 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:05.526+0000 7fb31e7a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:05.532 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:05.526+0000 7fb31e7a8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:05.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:51:05.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:51:05.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:51:05.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:51:05.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:51:05.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:05.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:51:05.723 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:51:05.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:05.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:51:05.916 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:05.910+0000 7fb31e7a8a40 -1 Falling back to public interface 2026-02-21T01:51:05.932 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:06.504 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:06.498+0000 7fb31e7a8a40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:51:06.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:06.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:06.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:51:06.934 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:51:06.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:06.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:51:07.146 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3842923015,v1:127.0.0.1:6811/3842923015] [v2:127.0.0.1:6812/3842923015,v1:127.0.0.1:6813/3842923015] exists,up 24ff08c5-1b74-4350-ade4-0abd133531ca 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:239: TEST_divergent_ec: run_osd td/divergent-priors 2 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/divergent-priors 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:51:07.147 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:07.148 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:51:07.149 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:51:07.150 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:51:07.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:51:07.151 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:51:07.151 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 14f76dc4-b360-4da2-ba08-d2303edbd7e6' 2026-02-21T01:51:07.151 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:51:07.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCLD5lpixapCRAAoaj2RZyQQuk3t6f3t2b9tQ== 2026-02-21T01:51:07.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCLD5lpixapCRAAoaj2RZyQQuk3t6f3t2b9tQ=="}' 2026-02-21T01:51:07.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 14f76dc4-b360-4da2-ba08-d2303edbd7e6 -i td/divergent-priors/2/new.json 2026-02-21T01:51:07.385 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:51:07.396 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/divergent-priors/2/new.json 2026-02-21T01:51:07.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCLD5lpixapCRAAoaj2RZyQQuk3t6f3t2b9tQ== --osd-uuid 14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:51:07.423 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:07.418+0000 7fa358d16a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:07.425 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:07.418+0000 7fa358d16a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:07.426 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:07.422+0000 7fa358d16a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:07.427 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:07.422+0000 7fa358d16a40 -1 bdev(0x56294ef9ac00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:51:07.427 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:07.422+0000 7fa358d16a40 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-21T01:51:08.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-21T01:51:08.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:51:08.070 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:51:08.071 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:51:08.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:51:08.358 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:51:08.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:51:08.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:51:08.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:51:08.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:51:08.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:51:08.389 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:08.382+0000 7f0384021a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:08.391 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:08.386+0000 7f0384021a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:08.392 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:08.386+0000 7f0384021a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:08.570 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:51:08.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:08.572 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:51:08.763 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:08.758+0000 7f0384021a40 -1 Falling back to public interface 2026-02-21T01:51:08.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:09.413 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:09.406+0000 7f0384021a40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:51:09.790 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:09.791 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:09.791 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:51:09.792 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:51:09.792 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:09.792 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:51:10.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:11.007 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:11.007 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:11.007 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:51:11.007 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:51:11.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:11.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:51:11.227 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2889610091,v1:127.0.0.1:6819/2889610091] [v2:127.0.0.1:6820/2889610091,v1:127.0.0.1:6821/2889610091] exists,up 14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:51:11.227 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:51:11.228 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:51:11.228 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:51:11.228 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:242: TEST_divergent_ec: ceph osd set noout 2026-02-21T01:51:11.518 INFO:tasks.workunit.client.0.trial009.stderr:noout is set 2026-02-21T01:51:11.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:243: TEST_divergent_ec: ceph osd set noin 2026-02-21T01:51:11.821 INFO:tasks.workunit.client.0.trial009.stderr:noin is set 2026-02-21T01:51:11.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:244: TEST_divergent_ec: ceph osd set nodown 2026-02-21T01:51:12.127 INFO:tasks.workunit.client.0.trial009.stderr:nodown is set 2026-02-21T01:51:12.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:245: TEST_divergent_ec: create_ec_pool test true k=2 m=1 2026-02-21T01:51:12.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2522: create_ec_pool: local pool_name=test 2026-02-21T01:51:12.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2523: create_ec_pool: shift 2026-02-21T01:51:12.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2524: create_ec_pool: local allow_overwrites=true 2026-02-21T01:51:12.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2525: create_ec_pool: shift 2026-02-21T01:51:12.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2527: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-21T01:51:12.395 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2529: create_ec_pool: create_pool test 1 1 erasure myprofile 2026-02-21T01:51:12.395 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create test 1 1 erasure myprofile 2026-02-21T01:51:12.766 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' already exists 2026-02-21T01:51:12.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T01:51:13.778 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2531: create_ec_pool: '[' true = true ']' 2026-02-21T01:51:13.778 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2532: create_ec_pool: ceph osd pool set test allow_ec_overwrites true 2026-02-21T01:51:14.089 INFO:tasks.workunit.client.0.trial009.stderr:set pool 1 allow_ec_overwrites to true 2026-02-21T01:51:14.103 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2535: create_ec_pool: wait_for_clean 2026-02-21T01:51:14.103 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:51:14.104 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:51:14.104 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:51:14.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:51:14.105 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:51:14.105 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:51:14.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:51:14.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:51:14.105 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:51:14.202 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:51:14.202 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:51:14.203 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:51:14.203 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:51:14.203 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:51:14.203 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:51:14.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:51:14.418 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:51:14.418 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:51:14.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:51:14.418 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:14.419 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:51:14.505 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836485 2026-02-21T01:51:14.505 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836485 2026-02-21T01:51:14.505 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485' 2026-02-21T01:51:14.505 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:14.506 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:51:14.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705668 2026-02-21T01:51:14.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705668 2026-02-21T01:51:14.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705668' 2026-02-21T01:51:14.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:14.588 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:51:14.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-21T01:51:14.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-21T01:51:14.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836485 1-38654705668 2-55834574852' 2026-02-21T01:51:14.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:14.667 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836485 2026-02-21T01:51:14.667 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:51:14.668 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836485 2026-02-21T01:51:14.668 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:14.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836485 2026-02-21T01:51:14.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836485' 2026-02-21T01:51:14.670 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836485 2026-02-21T01:51:14.670 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:14.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836485 2026-02-21T01:51:14.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:14.892 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705668 2026-02-21T01:51:14.892 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:14.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:51:14.894 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705668 2026-02-21T01:51:14.894 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:14.895 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705668 2026-02-21T01:51:14.896 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705668 2026-02-21T01:51:14.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705668' 2026-02-21T01:51:14.897 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:51:15.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705668 2026-02-21T01:51:15.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:15.115 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574852 2026-02-21T01:51:15.115 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:15.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:51:15.116 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574852 2026-02-21T01:51:15.117 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:15.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-21T01:51:15.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574852' 2026-02-21T01:51:15.119 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574852 2026-02-21T01:51:15.119 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:51:15.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-21T01:51:15.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:51:15.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:15.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:15.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:51:15.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:15.603 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:15.604 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:15.604 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:15.604 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:15.604 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:15.604 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:15.810 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:51:15.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:15.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:15.811 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2536: create_ec_pool: return 0 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:247: TEST_divergent_ec: flush_pg_stats 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:51:16.086 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:51:16.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:51:16.307 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:51:16.308 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:51:16.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:51:16.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:16.308 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:51:16.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836487 2026-02-21T01:51:16.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836487 2026-02-21T01:51:16.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487' 2026-02-21T01:51:16.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:16.388 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:51:16.470 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705670 2026-02-21T01:51:16.471 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705670 2026-02-21T01:51:16.471 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705670' 2026-02-21T01:51:16.471 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:16.471 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:51:16.553 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-21T01:51:16.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-21T01:51:16.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836487 1-38654705670 2-55834574855' 2026-02-21T01:51:16.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:16.554 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836487 2026-02-21T01:51:16.555 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:16.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:51:16.556 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836487 2026-02-21T01:51:16.556 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:16.558 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836487 2026-02-21T01:51:16.558 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-21T01:51:16.558 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836487 2026-02-21T01:51:16.559 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:16.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836485 -lt 21474836487 2026-02-21T01:51:16.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:51:17.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:51:17.775 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:17.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-21T01:51:17.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:17.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705670 2026-02-21T01:51:17.990 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:17.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:51:17.992 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705670 2026-02-21T01:51:17.992 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:17.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705670 2026-02-21T01:51:17.993 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705670 2026-02-21T01:51:17.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705670' 2026-02-21T01:51:17.994 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:51:18.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705670 -lt 38654705670 2026-02-21T01:51:18.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:18.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574855 2026-02-21T01:51:18.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:18.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:51:18.223 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574855 2026-02-21T01:51:18.223 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:18.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-21T01:51:18.225 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574855 2026-02-21T01:51:18.225 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-21T01:51:18.226 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:51:18.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-21T01:51:18.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:248: TEST_divergent_ec: wait_for_clean 2026-02-21T01:51:18.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:51:18.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:51:18.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:51:18.446 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:51:18.447 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:51:18.447 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:51:18.447 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:51:18.448 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:51:18.448 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:51:18.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:51:18.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:51:18.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:51:18.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:51:18.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:51:18.547 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:51:18.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:51:18.763 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:51:18.763 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:51:18.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:51:18.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:18.764 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:51:18.845 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836489 2026-02-21T01:51:18.845 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836489 2026-02-21T01:51:18.846 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489' 2026-02-21T01:51:18.846 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:18.846 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:51:18.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705673 2026-02-21T01:51:18.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705673 2026-02-21T01:51:18.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705673' 2026-02-21T01:51:18.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:18.926 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:51:19.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-21T01:51:19.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-21T01:51:19.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836489 1-38654705673 2-55834574857' 2026-02-21T01:51:19.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:19.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836489 2026-02-21T01:51:19.007 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:19.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:51:19.009 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836489 2026-02-21T01:51:19.009 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:19.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836489 2026-02-21T01:51:19.010 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836489 2026-02-21T01:51:19.011 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-21T01:51:19.011 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:19.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-21T01:51:19.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:51:20.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:51:20.233 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:20.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-21T01:51:20.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:51:21.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T01:51:21.449 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:21.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836489 2026-02-21T01:51:21.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:21.671 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705673 2026-02-21T01:51:21.672 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:21.673 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:51:21.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705673 2026-02-21T01:51:21.673 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:21.675 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705673 2026-02-21T01:51:21.675 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705673 2026-02-21T01:51:21.676 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705673' 2026-02-21T01:51:21.676 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:51:21.889 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705673 -lt 38654705673 2026-02-21T01:51:21.889 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:21.890 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574857 2026-02-21T01:51:21.890 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:21.890 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:51:21.891 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574857 2026-02-21T01:51:21.891 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:21.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-21T01:51:21.893 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574857 2026-02-21T01:51:21.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-21T01:51:21.893 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:51:22.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574857 2026-02-21T01:51:22.123 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:51:22.123 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:22.123 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:22.413 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:51:22.413 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:22.413 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:22.413 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:22.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:22.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:22.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:22.414 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:22.620 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:51:22.621 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:22.621 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:22.621 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:22.907 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:51:22.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:51:22.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:51:22.909 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:251: TEST_divergent_ec: ceph pg dump pgs --format=json 2026-02-21T01:51:22.909 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:251: TEST_divergent_ec: jq '.pg_stats[0].up_primary' 2026-02-21T01:51:23.117 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:51:23.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:251: TEST_divergent_ec: local divergent=1 2026-02-21T01:51:23.127 INFO:tasks.workunit.client.0.trial009.stdout:primary and soon to be divergent is 1 2026-02-21T01:51:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: echo 'primary and soon to be divergent is 1' 2026-02-21T01:51:23.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:253: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:51:23.334 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:51:23.335 INFO:tasks.workunit.client.0.trial009.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-21T01:51:13.394518+0000 0'0 27:18 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:11:56.774456+0000 0 0 2026-02-21T01:51:23.335 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:51:23.335 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:51:23.335 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:254: TEST_divergent_ec: local non_divergent= 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:257: TEST_divergent_ec: '[' 0 = 1 ']' 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:260: TEST_divergent_ec: non_divergent=' 0' 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:257: TEST_divergent_ec: '[' 1 = 1 ']' 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: continue 2026-02-21T01:51:23.346 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:23.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:257: TEST_divergent_ec: '[' 2 = 1 ']' 2026-02-21T01:51:23.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:260: TEST_divergent_ec: non_divergent=' 0 2' 2026-02-21T01:51:23.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:263: TEST_divergent_ec: echo 'writing initial objects' 2026-02-21T01:51:23.347 INFO:tasks.workunit.client.0.trial009.stdout:writing initial objects 2026-02-21T01:51:23.348 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: seq 1 100 2026-02-21T01:51:23.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_1 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.370 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.370 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_2 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_3 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.421 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_4 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_5 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.478 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_6 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.507 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.507 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_7 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_8 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.558 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.558 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_9 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.585 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_10 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.614 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.614 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_11 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_12 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_13 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.700 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.700 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_14 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.728 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_15 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.756 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.757 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_16 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_17 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.812 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.812 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_18 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_19 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.869 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.869 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_20 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_21 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_22 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_23 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:23.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:23.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_24 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_25 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_26 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_27 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_28 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_29 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.148 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_30 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_31 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_32 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_33 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.261 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.261 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_34 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_35 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_36 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_37 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_38 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.400 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.400 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_39 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.428 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_40 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_41 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.484 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_42 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_43 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.539 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_44 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.567 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.567 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_45 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_46 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.624 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.624 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_47 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_48 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_49 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_50 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.736 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_51 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_52 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.791 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.791 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_53 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.819 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_54 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_55 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.877 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_56 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.897 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.897 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_57 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_58 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.943 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.943 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_59 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_60 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:24.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:24.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_61 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_62 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.055 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.055 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_63 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_64 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.111 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.111 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_65 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_66 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_67 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.195 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.195 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_68 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.223 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_69 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_70 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_71 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.299 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_72 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_73 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_74 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_75 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_76 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_77 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_78 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.488 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.488 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_79 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.516 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.516 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_80 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.544 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_81 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.572 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.572 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_82 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_83 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.619 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.619 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_84 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_85 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.661 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.661 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_86 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.692 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.692 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_87 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_88 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_89 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_90 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_91 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.831 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.832 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_92 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.852 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.852 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_93 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_94 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_95 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.936 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.936 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_96 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_97 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:25.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:25.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_98 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:26.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:26.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_99 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:26.039 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:265: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-21T01:51:26.039 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:267: TEST_divergent_ec: rados -p test put existing_100 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:26.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:270: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:51:26.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:270: TEST_divergent_ec: wait_for_clean 2026-02-21T01:51:26.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:51:26.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:51:26.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:51:26.068 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:51:26.069 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:51:26.069 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:51:26.069 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:51:26.069 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:51:26.069 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:51:26.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:51:26.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:51:26.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:51:26.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:51:26.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:51:26.141 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:51:26.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:51:26.358 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:51:26.358 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:51:26.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:51:26.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:26.359 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:51:26.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836493 2026-02-21T01:51:26.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836493 2026-02-21T01:51:26.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493' 2026-02-21T01:51:26.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:26.441 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:51:26.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705676 2026-02-21T01:51:26.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705676 2026-02-21T01:51:26.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705676' 2026-02-21T01:51:26.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:26.525 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:51:26.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574861 2026-02-21T01:51:26.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574861 2026-02-21T01:51:26.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836493 1-38654705676 2-55834574861' 2026-02-21T01:51:26.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:26.605 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836493 2026-02-21T01:51:26.605 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:26.606 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:51:26.607 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836493 2026-02-21T01:51:26.607 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:26.608 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836493 2026-02-21T01:51:26.609 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 21474836493 2026-02-21T01:51:26.609 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836493' 2026-02-21T01:51:26.609 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:26.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836491 -lt 21474836493 2026-02-21T01:51:26.828 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:51:27.828 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:51:27.829 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:28.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836493 2026-02-21T01:51:28.049 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:28.049 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705676 2026-02-21T01:51:28.050 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:28.051 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:51:28.052 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705676 2026-02-21T01:51:28.052 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:28.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705676 2026-02-21T01:51:28.053 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 38654705676 2026-02-21T01:51:28.054 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705676' 2026-02-21T01:51:28.054 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:51:28.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705676 -lt 38654705676 2026-02-21T01:51:28.271 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:28.271 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574861 2026-02-21T01:51:28.271 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:28.273 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:51:28.273 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574861 2026-02-21T01:51:28.274 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:28.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574861 2026-02-21T01:51:28.275 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 55834574861 2026-02-21T01:51:28.275 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574861' 2026-02-21T01:51:28.276 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:51:28.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-21T01:51:28.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:51:28.487 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:28.487 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:28.765 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:51:28.765 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:28.766 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:28.766 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:28.766 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:28.766 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:28.766 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:28.766 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:28.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:51:28.970 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:28.971 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:28.971 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:29.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:51:29.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:51:29.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:51:29.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:272: TEST_divergent_ec: get_pg test existing_1 2026-02-21T01:51:29.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1084: get_pg: local poolname=test 2026-02-21T01:51:29.257 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1085: get_pg: local objectname=existing_1 2026-02-21T01:51:29.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: ceph --format json osd map test existing_1 2026-02-21T01:51:29.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1087: get_pg: jq -r .pgid 2026-02-21T01:51:29.475 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:272: TEST_divergent_ec: local pgid=1.0 2026-02-21T01:51:29.476 INFO:tasks.workunit.client.0.trial009.stdout:blackholing osds 0 2 2026-02-21T01:51:29.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:275: TEST_divergent_ec: echo 'blackholing osds 0 2' 2026-02-21T01:51:29.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:276: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:51:29.670 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:51:29.670 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 0 0 0 51200 0 0 16 84 16 active+clean 2026-02-21T01:51:13.394518+0000 27'100 27:118 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T04:11:56.774456+0000 0 0 2026-02-21T01:51:29.670 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:51:29.670 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:51:29.670 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:51:29.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: get_asok_path osd.0 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:29.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:51:29.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: CEPH_ARGS= 2026-02-21T01:51:29.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-21T01:51:29.733 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:51:29.733 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:51:29.733 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:51:29.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:29.739 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: get_asok_path osd.2 2026-02-21T01:51:29.739 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:51:29.739 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:51:29.740 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:51:29.740 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:29.740 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:29.740 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:51:29.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: CEPH_ARGS= 2026-02-21T01:51:29.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:279: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-21T01:51:29.790 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:51:29.791 INFO:tasks.workunit.client.0.trial009.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-21T01:51:29.791 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:51:29.796 INFO:tasks.workunit.client.0.trial009.stdout:writing divergent object 2026-02-21T01:51:29.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:283: TEST_divergent_ec: echo 'writing divergent object' 2026-02-21T01:51:29.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:285: TEST_divergent_ec: sleep 1 2026-02-21T01:51:29.797 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:284: TEST_divergent_ec: rados -p test put existing_100 /tmp/tmp.3wgIKZgB5J 2026-02-21T01:51:30.798 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:287: TEST_divergent_ec: rados -p test mksnap snap1 2026-02-21T01:51:30.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:286: TEST_divergent_ec: rados -p test put existing_100 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:30.870 INFO:tasks.workunit.client.0.trial009.stdout:created pool test snap snap1 2026-02-21T01:51:30.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: sleep 10 2026-02-21T01:51:30.873 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: expr 100 - 1 2026-02-21T01:51:30.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: rados -p test put existing_99 /tmp/tmp.S3GfTrhjDp 2026-02-21T01:51:40.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:290: TEST_divergent_ec: killall -9 rados 2026-02-21T01:51:40.920 INFO:tasks.workunit.client.0.trial009.stderr:rados: no process found 2026-02-21T01:51:40.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:293: TEST_divergent_ec: echo 'killing all the osds' 2026-02-21T01:51:40.920 INFO:tasks.workunit.client.0.trial009.stdout:killing all the osds 2026-02-21T01:51:40.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:294: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:51:41.132 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:51:41.132 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 active+clean 2026-02-21T01:51:13.394518+0000 28'104 28:124 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:41:21.161960+0000 0 0 2026-02-21T01:51:41.132 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:51:41.132 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:51:41.132 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:51:41.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:295: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd 2026-02-21T01:51:41.141 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:51:41.141 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:51:41.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:51:41.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:51:41.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:51:41.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:51:41.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:41.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:298: TEST_divergent_ec: ceph osd down osd.0 2026-02-21T01:51:41.716 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:51:41.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:41.727 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:298: TEST_divergent_ec: ceph osd down osd.1 2026-02-21T01:51:41.998 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:51:42.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: for i in $osds 2026-02-21T01:51:42.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:298: TEST_divergent_ec: ceph osd down osd.2 2026-02-21T01:51:42.284 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:51:42.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:300: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:42.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:302: TEST_divergent_ec: ceph osd out osd.0 2026-02-21T01:51:42.562 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already out. 2026-02-21T01:51:42.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:300: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:42.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:302: TEST_divergent_ec: ceph osd out osd.2 2026-02-21T01:51:42.852 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already out. 2026-02-21T01:51:42.862 INFO:tasks.workunit.client.0.trial009.stdout:bringing up non_divergent 0 2 2026-02-21T01:51:42.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:306: TEST_divergent_ec: echo 'bringing up non_divergent 0 2' 2026-02-21T01:51:42.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:307: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:51:43.077 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:51:43.077 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 stale+active+clean 2026-02-21T01:51:13.394518+0000 28'104 28:124 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:41:21.161960+0000 0 0 2026-02-21T01:51:43.077 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:51:43.077 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:51:43.077 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:51:43.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:310: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:51:43.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:43.089 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:51:43.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:51:43.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:51:43.092 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:51:43.092 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:51:43.093 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:51:43.110 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:43.106+0000 7f2f37092a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:43.112 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:43.106+0000 7f2f37092a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:43.112 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:43.106+0000 7f2f37092a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:43.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:51:43.303 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:51:43.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:43.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:51:43.479 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:43.474+0000 7f2f37092a40 -1 Falling back to public interface 2026-02-21T01:51:43.515 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:44.202 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:44.198+0000 7f2f37092a40 -1 osd.0 27 log_to_monitors true 2026-02-21T01:51:44.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:44.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:44.517 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:51:44.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:51:44.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:44.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:51:44.737 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6800/2905649494,v1:127.0.0.1:6801/2905649494] [v2:127.0.0.1:6802/2905649494,v1:127.0.0.1:6803/2905649494] exists,up bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:51:44.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:51:44.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:51:44.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:310: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:51:44.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:51:44.743 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:51:44.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:51:44.745 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:51:44.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:51:44.746 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-21T01:51:44.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:51:44.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:51:44.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:51:44.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:51:44.762 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:44.758+0000 7f47a78c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:44.763 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:44.758+0000 7f47a78c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:44.764 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:44.758+0000 7f47a78c5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:51:44.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:51:44.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:51:44.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:51:44.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:51:44.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:51:44.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:44.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:51:44.955 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:51:44.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:44.956 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:51:45.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:51:45.195 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:45.190+0000 7f47a78c5a40 -1 Falling back to public interface 2026-02-21T01:51:45.925 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:51:45.918+0000 7f47a78c5a40 -1 osd.2 27 log_to_monitors true 2026-02-21T01:51:46.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:51:46.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:51:46.174 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:51:46.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:51:46.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:51:46.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:51:46.387 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up out weight 0 up_from 38 up_thru 0 down_at 31 last_clean_interval [13,27) [v2:127.0.0.1:6810/3183459294,v1:127.0.0.1:6811/3183459294] [v2:127.0.0.1:6812/3183459294,v1:127.0.0.1:6813/3183459294] exists,up 14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:51:46.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:51:46.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:51:46.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:51:46.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:312: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:46.388 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:314: TEST_divergent_ec: ceph osd in osd.0 2026-02-21T01:51:46.636 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already in. 2026-02-21T01:51:46.646 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:312: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:51:46.646 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:314: TEST_divergent_ec: ceph osd in osd.2 2026-02-21T01:51:46.903 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already in. 2026-02-21T01:51:46.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:317: TEST_divergent_ec: sleep 5 2026-02-21T01:51:51.916 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:321: TEST_divergent_ec: expr 2 + 2 2026-02-21T01:51:51.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:321: TEST_divergent_ec: objname=existing_4 2026-02-21T01:51:51.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: echo 'writing non-divergent object existing_4' 2026-02-21T01:51:51.918 INFO:tasks.workunit.client.0.trial009.stdout:writing non-divergent object existing_4 2026-02-21T01:51:51.918 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:323: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:51:52.115 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:51:52.115 INFO:tasks.workunit.client.0.trial009.stdout:1.0 100 0 100 0 0 51200 0 0 16 84 16 active+undersized+degraded 2026-02-21T01:51:46.952519+0000 27'100 41:127 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T10:40:15.085855+0000 0 0 2026-02-21T01:51:52.115 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:51:52.115 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:51:52.115 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:51:52.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:324: TEST_divergent_ec: rados -p test put existing_4 /tmp/tmp.3wgIKZgB5J 2026-02-21T01:51:52.153 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:325: TEST_divergent_ec: rm -f /tmp/tmp.3wgIKZgB5J 2026-02-21T01:51:52.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:51:52.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: wait_for_clean 2026-02-21T01:51:52.155 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:51:52.155 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:51:52.155 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:51:52.156 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:51:52.156 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:51:52.156 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:51:52.156 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:51:52.156 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:51:52.156 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:51:52.228 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:51:52.229 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:51:52.229 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:51:52.229 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:51:52.229 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:51:52.229 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:51:52.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:51:52.453 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:51:52.453 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:51:52.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:51:52.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:52.454 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:51:52.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-21T01:51:52.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-21T01:51:52.535 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855364' 2026-02-21T01:51:52.535 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:52.535 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:51:52.592 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:51:52.595 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:51:52.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:51:52.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:51:52.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:51:52.596 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:51:52.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=163208757252 2026-02-21T01:51:52.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 163208757252 2026-02-21T01:51:52.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-150323855364 2-163208757252' 2026-02-21T01:51:52.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:52.679 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-150323855364 2026-02-21T01:51:52.679 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:52.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:51:52.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-150323855364 2026-02-21T01:51:52.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:52.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855364 2026-02-21T01:51:52.683 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 150323855364 2026-02-21T01:51:52.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 150323855364' 2026-02-21T01:51:52.683 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:51:52.889 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-21T01:51:52.889 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:51:52.890 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-163208757252 2026-02-21T01:51:52.890 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:51:52.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:51:52.892 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-163208757252 2026-02-21T01:51:52.892 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:51:52.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=163208757252 2026-02-21T01:51:52.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 163208757252' 2026-02-21T01:51:52.894 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 163208757252 2026-02-21T01:51:52.894 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:51:53.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 163208757252 -lt 163208757252 2026-02-21T01:51:53.125 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:51:53.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:53.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:53.414 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:51:53.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:53.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:53.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:53.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:53.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:53.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:53.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:53.627 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:51:53.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:53.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:53.628 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:53.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:51:53.915 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:51:53.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:51:53.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:51:53.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:51:53.916 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:51:54.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:51:54.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:54.017 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:54.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:54.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:54.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:54.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:54.018 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:54.229 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:51:54.230 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:54.230 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:54.230 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:51:54.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:51:54.519 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:51:54.519 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:51:54.799 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:51:54.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:51:54.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:51:54.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:51:54.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:51:55.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:51:55.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:55.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:55.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:55.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:55.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:55.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:55.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:55.209 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:51:55.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:55.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:55.210 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:51:55.486 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:51:55.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:51:55.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:51:55.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:51:55.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:51:55.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:51:56.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:51:56.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:56.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:56.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:56.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:56.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:56.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:56.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:56.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:51:56.375 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:56.376 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:56.376 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:51:56.649 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:51:56.650 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:51:56.650 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:51:56.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:51:56.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:51:56.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:51:56.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:51:56.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:51:57.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:51:57.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:51:57.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:51:57.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:51:57.725 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:51:57.726 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:51:57.726 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:51:57.726 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:51:57.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:51:57.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:51:57.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:51:57.937 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:51:58.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:51:58.213 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:51:58.213 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:51:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:51:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:51:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:51:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:51:58.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:52:00.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:00.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:00.087 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:00.087 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:00.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:00.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:00.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:00.088 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:00.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:00.304 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:00.304 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:00.304 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:00.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:00.588 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:00.588 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:00.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:00.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:00.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:52:00.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:00.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:52:04.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:04.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:04.076 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:04.076 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:04.076 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:04.077 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:04.077 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:04.077 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:04.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:04.295 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:04.295 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:04.295 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:04.572 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:04.572 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:04.572 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:04.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:04.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:04.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:04.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:04.573 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:04.573 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:52:04.852 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:04.852 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:52:11.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:11.252 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:11.253 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:11.253 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:11.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:11.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:11.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:11.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:11.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:11.461 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:11.461 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:11.461 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:11.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:11.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:11.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:11.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:11.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:11.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:11.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:11.741 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:11.741 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:12.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:12.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:12.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:52:12.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:12.019 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:52:19.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:19.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:19.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:19.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:19.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:19.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:19.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:19.321 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:19.527 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:19.527 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:19.528 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:19.528 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:19.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:19.810 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:19.810 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:20.097 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:20.097 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:20.097 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:52:20.097 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:52:20.366 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:52:20.366 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "842eb730-4353-44d6-b6c2-cb85b6020a70", 2026-02-21T01:52:20.366 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:52:20.362726+0000", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:52:20.367 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.368 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:52:20.369 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:52:20.370 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:51:00.276120Z", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:51:00.276120Z", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:52:20.371 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:52:20.372 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:20.373 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:52:20.374 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 41, 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:51:00.348293+0000", 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:51:46.948737+0000", 2026-02-21T01:52:20.375 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:51:46.076941+0000", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:51:46.846747+0000", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:52:20.376 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 10, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:52:20.377 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:51:12.619927+0000", 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 16389, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "type": 3, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 1, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:52:20.378 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "off", 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 1, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 1, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 1, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 1, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:52:20.379 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "28", 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "0", 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "pool", 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 1, 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 28, 2026-02-21T01:52:20.380 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [ 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "snapid": 1, 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "stamp": "2026-02-21T01:51:30.815822+0000", 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "name": "snap1" 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:52:20.381 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "myprofile", 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:52:20.382 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 8192, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:52:20.383 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {} 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "bbfde735-2cd5-4c97-af04-f608bbd9a42f", 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 5, 2026-02-21T01:52:20.384 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 27, 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 35, 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 40, 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 29, 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.385 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:52:20.386 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.387 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:52:20.388 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 2905649494 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.389 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/2905649494", 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/2905649494", 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/2905649494", 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/2905649494", 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "24ff08c5-1b74-4350-ade4-0abd133531ca", 2026-02-21T01:52:20.390 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 9, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 23, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 30, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.391 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.392 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.393 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.394 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:20.395 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/3842923015", 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/3842923015", 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3842923015", 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3842923015", 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:20.396 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "14f76dc4-b360-4da2-ba08-d2303edbd7e6", 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 13, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 27, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 38, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 31, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:20.397 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.398 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.399 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.400 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3183459294 2026-02-21T01:52:20.401 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/3183459294", 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/3183459294", 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3183459294", 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3183459294", 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.402 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:51:41.658358+0000", 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:51:04.083691+0000", 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:20.403 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:51:41.938019+0000", 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:51:42.227727+0000", 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:52:20.404 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:51:10.365757+0000", 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:52:20.405 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "myprofile": { 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "crush-device-class": "", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "crush-failure-domain": "osd", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "crush-num-failure-domains": "0", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "crush-osds-per-failure-domain": "0", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "crush-root": "default", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "m": "1", 2026-02-21T01:52:20.406 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:52:20.407 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/2905649494,v1:127.0.0.1:6803/2905649494]", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:52:20.408 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:20.409 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:51:01.337133Z", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/2905649494,v1:127.0.0.1:6801/2905649494]", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/2905649494,v1:127.0.0.1:6807/2905649494]", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/2905649494,v1:127.0.0.1:6805/2905649494]", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:52:20.410 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/3842923015,v1:127.0.0.1:6813/3842923015]", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:52:20.411 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:52:20.412 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:51:05.118129Z", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/3842923015,v1:127.0.0.1:6811/3842923015]", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3842923015,v1:127.0.0.1:6817/3842923015]", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3842923015,v1:127.0.0.1:6815/3842923015]", 2026-02-21T01:52:20.413 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/3183459294,v1:127.0.0.1:6813/3183459294]", 2026-02-21T01:52:20.414 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:52:20.415 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:51:07.969764Z", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/3183459294,v1:127.0.0.1:6811/3183459294]", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3183459294,v1:127.0.0.1:6817/3183459294]", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3183459294,v1:127.0.0.1:6815/3183459294]", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:52:20.416 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 0, 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [] 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.417 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 35 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 41 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 41, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.418 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.419 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.420 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:52:20.421 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:52:20.422 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:20.423 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:20.424 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:52:20.425 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 1, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "test", 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "type": 3, 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "op": "set_chooseleaf_tries", 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: "num": 5 2026-02-21T01:52:20.426 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "op": "set_choose_tries", 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "num": 100 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_indep", 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.427 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stderr:report 831092194 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:52:20.428 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 1, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:51:00:348072+0000", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:52:20.429 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 1, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:52:20.430 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 52200, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 101, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 1, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 303, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 101, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 101, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:52:20.431 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 18, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 18, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "up": 3, 2026-02-21T01:52:20.432 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 3, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83892, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2116, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 24, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81703, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488908, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036641792, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2166784, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 1536419, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 25114, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83664358 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:52:20.433 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 83892, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2116, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 24, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 81703, 2026-02-21T01:52:20.434 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314488908, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322036641792, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2166784, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 1536419, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 25114, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 83664358 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.435 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 52200, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 101, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 1, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 303, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 101, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 101, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:52:20.436 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1241088, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 1241088, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 18, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 18, 2026-02-21T01:52:20.437 INFO:tasks.workunit.client.0.trial009.stdout: "up": 3, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 3, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 150323855369 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 38654705679 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 163208757257 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "num": 1 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:52:20.438 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2147483647, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 1, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:20.439 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 3, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:52:08.794214+0000", 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "services": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "osd": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "daemons": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "summary": "", 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "0": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "2": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 40, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 231, 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:52:20.440 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 323, 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.0 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:52:20.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:52:20.484 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 0 --op log --pgid 1.0 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=0 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/0 2026-02-21T01:52:20.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/0 --op log --pgid 1.0 2026-02-21T01:52:20.670 INFO:tasks.workunit.client.0.trial009.stderr:PG '1.0' not found 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:52:20.771 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/0 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:52:20.772 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:52:20.773 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:52:20.773 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/0 2026-02-21T01:52:20.774 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:52:20.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:52:20.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:52:20.775 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/0/whoami 2026-02-21T01:52:20.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:52:20.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:52:20.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:52:20.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:52:20.794 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:20.789+0000 7f924b9cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:20.795 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:20.789+0000 7f924b9cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:20.796 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:20.789+0000 7f924b9cea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:20.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:52:20.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:52:20.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:52:20.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:52:20.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:52:20.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:52:20.985 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:52:20.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:52:20.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:52:20.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:52:21.197 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 35 up_thru 40 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6800/2905649494,v1:127.0.0.1:6801/2905649494] [v2:127.0.0.1:6802/2905649494,v1:127.0.0.1:6803/2905649494] exists,up bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:52:21.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:52:21.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:52:21.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:52:21.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:52:21.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.2 2026-02-21T01:52:21.199 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:52:21.199 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:52:21.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:52:21.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:52:21.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:52:21.200 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:21.193+0000 7f924b9cea40 -1 Falling back to public interface 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 2 --op log --pgid 1.0 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/2 2026-02-21T01:52:21.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/2 --op log --pgid 1.0 2026-02-21T01:52:21.470 INFO:tasks.workunit.client.0.trial009.stderr:PG '1.0' not found 2026-02-21T01:52:21.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-21T01:52:21.587 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/2 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-21T01:52:21.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:52:21.589 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:52:21.591 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/2 2026-02-21T01:52:21.592 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/2/whoami 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:52:21.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:52:21.607 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:21.601+0000 7f1ca21afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:21.609 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:21.601+0000 7f1ca21afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:21.609 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:21.601+0000 7f1ca21afa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:52:21.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:52:21.809 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:52:21.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:52:21.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:52:22.027 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 38 up_thru 0 down_at 31 last_clean_interval [13,27) [v2:127.0.0.1:6810/3183459294,v1:127.0.0.1:6811/3183459294] [v2:127.0.0.1:6812/3183459294,v1:127.0.0.1:6813/3183459294] exists,up 14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:52:22.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:52:22.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:52:22.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:52:22.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:336: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 1 --op log --pgid 1.0 2026-02-21T01:52:22.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-21T01:52:22.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-21T01:52:22.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-21T01:52:22.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-21T01:52:22.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-21T01:52:22.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op log --pgid 1.0 2026-02-21T01:52:22.077 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:22.069+0000 7f924b9cea40 -1 osd.0 41 log_to_monitors true 2026-02-21T01:52:22.091 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:22.085+0000 7f1ca21afa40 -1 Falling back to public interface 2026-02-21T01:52:22.260 INFO:tasks.workunit.client.0.trial009.stderr:PG '1.0' not found 2026-02-21T01:52:22.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1256: _objectstore_tool_nodown: return 1 2026-02-21T01:52:22.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-21T01:52:22.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: wait_for_clean 2026-02-21T01:52:22.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:52:22.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:52:22.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:52:22.360 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 20 .1 2026-02-21T01:52:22.360 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:52:22.360 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:52:22.360 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:52:22.360 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:52:22.360 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:52:22.423 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-21T01:52:22.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:52:22.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:52:22.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:52:22.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:52:22.424 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:52:22.640 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:52:22.640 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:52:22.640 INFO:tasks.workunit.client.0.trial009.stderr:2' 2026-02-21T01:52:22.640 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:52:22.640 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:52:22.641 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:52:22.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561027 2026-02-21T01:52:22.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561027 2026-02-21T01:52:22.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-188978561027' 2026-02-21T01:52:22.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:52:22.723 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:52:22.785 INFO:tasks.workunit.client.0.trial009.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-21T01:52:22.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-21T01:52:22.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-21T01:52:22.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-21T01:52:22.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:52:22.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:52:22.940 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:22.933+0000 7f1ca21afa40 -1 osd.2 41 log_to_monitors true 2026-02-21T01:52:27.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=206158430211 2026-02-21T01:52:27.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 206158430211 2026-02-21T01:52:27.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-188978561027 2-206158430211' 2026-02-21T01:52:27.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:52:27.877 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-188978561027 2026-02-21T01:52:27.877 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:52:27.878 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:52:27.878 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-188978561027 2026-02-21T01:52:27.878 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:52:27.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561027 2026-02-21T01:52:27.880 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 188978561027 2026-02-21T01:52:27.881 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 188978561027' 2026-02-21T01:52:27.881 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:52:28.095 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561027 -lt 188978561027 2026-02-21T01:52:28.096 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:52:28.096 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-206158430211 2026-02-21T01:52:28.096 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:52:28.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:52:28.098 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-206158430211 2026-02-21T01:52:28.098 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:52:28.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=206158430211 2026-02-21T01:52:28.100 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 206158430211 2026-02-21T01:52:28.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 206158430211' 2026-02-21T01:52:28.101 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:52:28.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 206158430211 2026-02-21T01:52:28.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:52:29.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:52:29.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:52:29.532 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 206158430212 -lt 206158430211 2026-02-21T01:52:29.533 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:52:29.533 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:29.534 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:29.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:52:29.820 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:29.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:29.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:29.821 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:29.822 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:29.822 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:29.822 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:30.031 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:30.032 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:30.032 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:30.032 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:30.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:30.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' -1 2026-02-21T01:52:30.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1665: wait_for_clean: loop=0 2026-02-21T01:52:30.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1666: wait_for_clean: num_active_clean=0 2026-02-21T01:52:30.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:30.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.1 2026-02-21T01:52:30.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:30.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:30.419 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:30.419 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:30.420 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:30.420 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:30.420 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:30.420 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:30.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:30.633 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:30.633 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:30.633 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:30.913 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:30.914 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:30.915 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:31.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:31.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:31.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 1 >= 8 )) 2026-02-21T01:52:31.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:31.188 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.2 2026-02-21T01:52:31.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:31.389 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:31.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:31.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:31.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:31.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:31.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:31.390 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:31.596 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:31.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:31.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:31.597 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:31.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:31.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:31.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:31.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:31.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:31.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:31.876 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:31.876 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:31.877 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:32.157 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:32.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:32.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 2 >= 8 )) 2026-02-21T01:52:32.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:32.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.4 2026-02-21T01:52:32.558 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:32.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:32.559 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:32.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:32.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:32.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:32.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:32.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:32.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:32.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:32.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:32.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:33.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:33.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:33.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:33.044 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:33.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:33.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:33.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 3 >= 8 )) 2026-02-21T01:52:33.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:33.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 0.8 2026-02-21T01:52:34.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:34.121 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:34.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:34.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:34.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:34.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:34.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:34.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:34.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:34.330 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:34.330 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:34.331 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:34.606 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:34.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:34.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:34.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:34.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:34.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:34.607 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:34.608 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:34.608 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:34.886 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:34.886 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:34.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 4 >= 8 )) 2026-02-21T01:52:34.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:34.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 1.6 2026-02-21T01:52:36.487 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:36.488 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:36.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:36.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:36.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:36.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:36.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:36.489 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:36.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:36.711 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:36.711 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:36.711 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:37.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:37.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:37.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:37.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:37.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:37.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:37.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:37.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:37.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:37.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:37.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:37.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 5 >= 8 )) 2026-02-21T01:52:37.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:37.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 3.2 2026-02-21T01:52:40.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:40.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:40.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:40.490 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:40.491 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:40.491 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:40.491 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:40.491 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:40.700 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:40.701 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:40.701 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:40.701 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:40.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:40.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:40.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:40.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:40.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:40.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:40.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:40.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:40.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:41.262 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:41.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:41.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 6 >= 8 )) 2026-02-21T01:52:41.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:41.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 6.4 2026-02-21T01:52:47.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:47.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:47.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:47.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:47.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:47.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:47.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:47.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:47.888 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:47.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:47.889 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:47.889 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:48.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:48.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:48.175 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:48.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:48.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:48.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 7 >= 8 )) 2026-02-21T01:52:48.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: eval 2026-02-21T01:52:48.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: sleep 7.3 2026-02-21T01:52:55.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: loop+=1 2026-02-21T01:52:55.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:52:55.768 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:52:55.768 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:52:55.768 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:52:55.768 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:52:55.768 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:52:55.768 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:52:55.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=0 2026-02-21T01:52:55.979 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:52:55.979 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:52:55.979 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:52:56.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 0 = 1 2026-02-21T01:52:56.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1664: wait_for_clean: test 0 '!=' 0 2026-02-21T01:52:56.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: get_is_making_recovery_progress 2026-02-21T01:52:56.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1325: get_is_making_recovery_progress: local recovery_progress 2026-02-21T01:52:56.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1326: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-21T01:52:56.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1327: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-21T01:52:56.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1328: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-21T01:52:56.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: ceph --format json status 2026-02-21T01:52:56.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-21T01:52:56.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1330: get_is_making_recovery_progress: local progress=null 2026-02-21T01:52:56.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1331: get_is_making_recovery_progress: test null '!=' null 2026-02-21T01:52:56.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1669: wait_for_clean: (( 8 >= 8 )) 2026-02-21T01:52:56.531 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1670: wait_for_clean: ceph report 2026-02-21T01:52:56.814 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:52:56.814 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_fingerprint": "842eb730-4353-44d6-b6c2-cb85b6020a70", 2026-02-21T01:52:56.814 INFO:tasks.workunit.client.0.trial009.stdout: "version": "20.2.0-681-g36d388f2", 2026-02-21T01:52:56.814 INFO:tasks.workunit.client.0.trial009.stdout: "commit": "36d388f27052dd940bdbaf543bb05bdba0acc155", 2026-02-21T01:52:56.814 INFO:tasks.workunit.client.0.trial009.stdout: "timestamp": "2026-02-21T01:52:56.810139+0000", 2026-02-21T01:52:56.814 INFO:tasks.workunit.client.0.trial009.stdout: "tag": "", 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "health": { 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "status": "HEALTH_WARN", 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "checks": { 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "OSDMAP_FLAGS": { 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "count": 17 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [], 2026-02-21T01:52:56.815 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "OSD_DOWN": { 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "message": "1 osds down", 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: "message": "osd.1 (root=default,host=trial009) is down" 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.816 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "PG_DEGRADED": { 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "severity": "HEALTH_WARN", 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "summary": { 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "count": 1 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "detail": [ 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.817 INFO:tasks.workunit.client.0.trial009.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: "muted": false 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: "mutes": [] 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_first_committed": 1, 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: "monmap_last_committed": 1, 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: "monmap": { 2026-02-21T01:52:56.818 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:51:00.276120Z", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:51:00.276120Z", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release": 20, 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "min_mon_release_name": "tentacle", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "election_strategy": 1, 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "disallowed_leaders": "", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": false, 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "tiebreaker_mon": "", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "removed_ranks": "", 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "features": { 2026-02-21T01:52:56.819 INFO:tasks.workunit.client.0.trial009.stdout: "persistent": [ 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "kraken", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "luminous", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "mimic", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap-prune", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "nautilus", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "octopus", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "pacific", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "elector-pinging", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "quincy", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "reef", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "squid", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "tentacle", 2026-02-21T01:52:56.820 INFO:tasks.workunit.client.0.trial009.stdout: "nvmeof_beacon_diff" 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "optional": [] 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "mons": [ 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "rank": 0, 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "name": "a", 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.821 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115", 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 0 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:7115/0", 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "priority": 0, 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 0, 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "crush_location": "{}" 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.822 INFO:tasks.workunit.client.0.trial009.stdout: "quorum": [ 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: 0 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap": { 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 49, 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "fsid": "2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0", 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "created": "2026-02-21T01:51:00.348293+0000", 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:52:23.338489+0000", 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "last_up_change": "2026-02-21T01:52:23.236532+0000", 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "last_in_change": "2026-02-21T01:51:46.846747+0000", 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "flags_num": 5800384, 2026-02-21T01:52:56.823 INFO:tasks.workunit.client.0.trial009.stdout: "flags_set": [ 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "nodown", 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "noin", 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "noout", 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "pglog_hardlimit", 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snapdirs", 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "recovery_deletes", 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "sortbitwise" 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "crush_version": 12, 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "full_ratio": 0.99000000953674316, 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-21T01:52:56.824 INFO:tasks.workunit.client.0.trial009.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_snapshot": "", 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "pool_max": 1, 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "max_osd": 3, 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "require_min_compat_client": "luminous", 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "min_compat_client": "jewel", 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "require_osd_release": "tentacle", 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "allow_crimson": false, 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "pools": [ 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "pool_name": "test", 2026-02-21T01:52:56.825 INFO:tasks.workunit.client.0.trial009.stdout: "create_time": "2026-02-21T01:51:12.619927+0000", 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "flags": 16389, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "type": 3, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "size": 3, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "min_size": 2, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "crush_rule": 1, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_count": 0, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_target": 0, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_barrier": 0, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "is_stretch_pool": false, 2026-02-21T01:52:56.826 INFO:tasks.workunit.client.0.trial009.stdout: "object_hash": 2, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "pg_autoscale_mode": "off", 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num": 1, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num": 1, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "pg_placement_num_target": 1, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_target": 1, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "pg_num_pending": 1, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "last_pg_merge_meta": { 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "source_pgid": "0.0", 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "ready_epoch": 0, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_started": 0, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": 0, 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "source_version": "0'0", 2026-02-21T01:52:56.827 INFO:tasks.workunit.client.0.trial009.stdout: "target_version": "0'0" 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "last_change": "28", 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend": "0", 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "last_force_op_resend_preluminous": "0", 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "auid": 0, 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "snap_mode": "pool", 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "snap_seq": 1, 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "snap_epoch": 28, 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "pool_snaps": [ 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "snapid": 1, 2026-02-21T01:52:56.828 INFO:tasks.workunit.client.0.trial009.stdout: "stamp": "2026-02-21T01:51:30.815822+0000", 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "name": "snap1" 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps": "[]", 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_bytes": 0, 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "quota_max_objects": 0, 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "tiers": [], 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "tier_of": -1, 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "read_tier": -1, 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "write_tier": -1, 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "cache_mode": "none", 2026-02-21T01:52:56.829 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_bytes": 0, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "target_max_objects": 0, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_flush_age": 0, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "cache_min_evict_age": 0, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profile": "myprofile", 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_params": { 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "type": "none" 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_period": 0, 2026-02-21T01:52:56.830 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_count": 0, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "use_gmt_hitset": true, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "min_read_recency_for_promote": 0, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "min_write_recency_for_promote": 0, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_grade_decay_rate": 0, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "hit_set_search_last_n": 0, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "grade_table": [], 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "stripe_width": 8192, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "expected_num_objects": 0, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "fast_read": false, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "nonprimary_shards": "{}", 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "options": {}, 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: "application_metadata": {} 2026-02-21T01:52:56.831 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "osds": [ 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "bbfde735-2cd5-4c97-af04-f608bbd9a42f", 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 35, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 41, 2026-02-21T01:52:56.832 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 44, 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 48, 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 43, 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6800", 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.833 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6801", 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6802", 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.834 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6803", 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.835 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6806", 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6807", 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.836 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.837 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.837 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6804", 2026-02-21T01:52:56.837 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.837 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.837 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.837 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6805", 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1283348058 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6801/1283348058", 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6803/1283348058", 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6807/1283348058", 2026-02-21T01:52:56.838 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6805/1283348058", 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "24ff08c5-1b74-4350-ade4-0abd133531ca", 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "up": 0, 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:52:56.839 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 0, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 0, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 9, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 23, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 30, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.840 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.841 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.842 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.843 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 3842923015 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/3842923015", 2026-02-21T01:52:56.844 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/3842923015", 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3842923015", 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3842923015", 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "exists" 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "uuid": "14f76dc4-b360-4da2-ba08-d2303edbd7e6", 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "up": 1, 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "in": 1, 2026-02-21T01:52:56.845 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 1, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "primary_affinity": 1, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_begin": 38, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "last_clean_end": 41, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 48, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "up_thru": 0, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "down_at": 47, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "lost_at": 0, 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "public_addrs": { 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6810", 2026-02-21T01:52:56.846 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6811", 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addrs": { 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.847 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6812", 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6813", 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addrs": { 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.848 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6816", 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6817", 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addrs": { 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: "addrvec": [ 2026-02-21T01:52:56.849 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v2", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6814", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "type": "v1", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "127.0.0.1:6815", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "nonce": 1812247902 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "public_addr": "127.0.0.1:6811/1812247902", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "cluster_addr": "127.0.0.1:6813/1812247902", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1812247902", 2026-02-21T01:52:56.850 INFO:tasks.workunit.client.0.trial009.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1812247902", 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "state": [ 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "exists", 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "up" 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "osd_xinfo": [ 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:52:22.196542+0000", 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:52:56.851 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:51:04.083691+0000", 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:51:41.938019+0000", 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.852 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "down_stamp": "2026-02-21T01:52:23.134259+0000", 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_probability": 0, 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "laggy_interval": 0, 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "features": 4544132024016699391, 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "old_weight": 0, 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "last_purged_snaps_scrub": "2026-02-21T01:51:10.365757+0000", 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "dead_epoch": 0 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap": [], 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_items": [], 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "pg_upmap_primaries": [], 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "pg_temp": [], 2026-02-21T01:52:56.853 INFO:tasks.workunit.client.0.trial009.stdout: "primary_temp": [], 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "blocklist": {}, 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "range_blocklist": {}, 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "erasure_code_profiles": { 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "default": { 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "m": "2", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "myprofile": { 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "crush-device-class": "", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "crush-failure-domain": "osd", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "crush-num-failure-domains": "0", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "crush-osds-per-failure-domain": "0", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "crush-root": "default", 2026-02-21T01:52:56.854 INFO:tasks.workunit.client.0.trial009.stdout: "k": "2", 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "m": "1", 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "plugin": "isa", 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "technique": "reed_sol_van" 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "removed_snaps_queue": [], 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "new_removed_snaps": [], 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "new_purged_snaps": [], 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "crush_node_flags": {}, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "device_class_flags": {}, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode": { 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_enabled": false, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_bucket_count": 0, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "degraded_stretch_mode": 0, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "recovering_stretch_mode": 0, 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: "stretch_mode_bucket": 0 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.855 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "osd_metadata": [ 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6802/1283348058,v1:127.0.0.1:6803/1283348058]", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:52:56.856 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:56.857 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:51:01.337133Z", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6800/1283348058,v1:127.0.0.1:6801/1283348058]", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6806/1283348058,v1:127.0.0.1:6807/1283348058]", 2026-02-21T01:52:56.858 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6804/1283348058,v1:127.0.0.1:6805/1283348058]", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/0", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:56.859 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/3842923015,v1:127.0.0.1:6813/3842923015]", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:52:56.860 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:51:05.118129Z", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/3842923015,v1:127.0.0.1:6811/3842923015]", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3842923015,v1:127.0.0.1:6817/3842923015]", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3842923015,v1:127.0.0.1:6815/3842923015]", 2026-02-21T01:52:56.861 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/1", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:52:56.862 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "arch": "x86_64", 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "back_addr": "[v2:127.0.0.1:6812/1812247902,v1:127.0.0.1:6813/1812247902]", 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "back_iface": "", 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs": "1", 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_db": "0", 2026-02-21T01:52:56.863 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_dedicated_wal": "0", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluefs_single_shared_device": "1", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocation_from_file": "1", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_allocator": "hybrid", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_access_mode": "file", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_block_size": "4096", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_rotational": "0", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_size": "107374182400", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_support_discard": "1", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_bdev_type": "ssd", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_min_alloc_size": "4096", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_onode_segmentation": "inactive", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "bluestore_write_mode": "classic", 2026-02-21T01:52:56.864 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_release": "tentacle", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_short": "20.2.0-681-g36d388f2", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "ceph_version_when_created": "ceph version 20.2.0-681-g36d388f2 (36d388f27052dd940bdbaf543bb05bdba0acc155) tentacle (stable - None)", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "created_at": "2026-02-21T01:51:07.969764Z", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "default_device_class": "ssd", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6977", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "devices": "nvme0n1", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "distro": "ubuntu", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "distro_version": "22.04", 2026-02-21T01:52:56.865 INFO:tasks.workunit.client.0.trial009.stdout: "front_addr": "[v2:127.0.0.1:6810/1812247902,v1:127.0.0.1:6811/1812247902]", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "front_iface": "", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1812247902,v1:127.0.0.1:6817/1812247902]", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1812247902,v1:127.0.0.1:6815/1812247902]", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "hostname": "trial009", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "journal_rotational": "0", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "mem_swap_kb": "8388604", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "mem_total_kb": "130961696", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_node": "0", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "objectstore_numa_nodes": "0", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "os": "Linux", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "osd_data": "td/divergent-priors/2", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "osd_objectstore": "bluestore", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "osdspec_affinity": "", 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "rotational": "0" 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_clean_epochs": { 2026-02-21T01:52:56.866 INFO:tasks.workunit.client.0.trial009.stdout: "min_last_epoch_clean": 24, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "last_epoch_clean": { 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "per_pool": [ 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "floor": 24 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "osd_epochs": [ 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 46 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 49 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_first_committed": 1, 2026-02-21T01:52:56.867 INFO:tasks.workunit.client.0.trial009.stdout: "osdmap_last_committed": 49, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "crushmap": { 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "devices": [ 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.0", 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.1", 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd.2", 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "class": "ssd" 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "types": [ 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 0, 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: "name": "osd" 2026-02-21T01:52:56.868 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "name": "host" 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 2, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "name": "chassis" 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 3, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "name": "rack" 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 4, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "name": "row" 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 5, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pdu" 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 6, 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: "name": "pod" 2026-02-21T01:52:56.869 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 7, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "name": "room" 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 8, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "name": "datacenter" 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 9, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "name": "zone" 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 10, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "name": "region" 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "name": "root" 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "buckets": [ 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.870 INFO:tasks.workunit.client.0.trial009.stdout: "id": -1, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "id": -2, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "name": "default~ssd", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 11, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "root", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.871 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "id": -3, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009", 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.872 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "id": -4, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "name": "trial009~ssd", 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "type_id": 1, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "type_name": "host", 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 19206, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "alg": "straw2", 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "hash": "rjenkins1", 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "items": [ 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "id": 0, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 0 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "id": 1, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 1 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.873 INFO:tasks.workunit.client.0.trial009.stdout: "id": 2, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "weight": 6402, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "pos": 2 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "rules": [ 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 0, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "replicated_rule", 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "type": 1, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_firstn", 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.874 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "rule_id": 1, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "rule_name": "test", 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "type": 3, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "steps": [ 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "op": "set_chooseleaf_tries", 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "num": 5 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "op": "set_choose_tries", 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "num": 100 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "op": "take", 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "item": -1, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "item_name": "default" 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "op": "choose_indep", 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "num": 0, 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: "type": "osd" 2026-02-21T01:52:56.875 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stderr:report 1014575559 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "op": "emit" 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: ] 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "tunables": { 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_tries": 0, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "choose_local_fallback_tries": 0, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "choose_total_tries": 50, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_descend_once": 1, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_vary_r": 1, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "chooseleaf_stable": 1, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "msr_descents": 100, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "msr_collision_tries": 100, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "straw_calc_version": 1, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "allowed_bucket_algs": 54, 2026-02-21T01:52:56.876 INFO:tasks.workunit.client.0.trial009.stdout: "profile": "jewel", 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "optimal_tunables": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "legacy_tunables": 0, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "minimum_required_version": "jewel", 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables2": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "has_v2_rules": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables3": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "has_v3_rules": 0, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "has_v4_buckets": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "require_feature_tunables5": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "has_v5_rules": 0, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "has_msr_rules": 0 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "choose_args": {} 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "fsmap": { 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "btime": "2026-02-21T01:51:00:348072+0000", 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "default_fscid": -1, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "compat": { 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "compat": {}, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "ro_compat": {}, 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "incompat": { 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "feature_1": "base v0.20", 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "feature_2": "client writeable ranges", 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "feature_3": "default file layouts on dirs", 2026-02-21T01:52:56.877 INFO:tasks.workunit.client.0.trial009.stdout: "feature_4": "dir inode in separate object", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_5": "mds uses versioned encoding", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_8": "no anchor table", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_9": "file layout v2", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_10": "snaprealm v2", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_11": "minor log segments", 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_12": "quiesce subvolumes" 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "feature_flags": { 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "enable_multiple": true, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "ever_enabled_multiple": true 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "standbys": [], 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "filesystems": [] 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_first_committed": 1, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "mdsmap_last_committed": 1, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "auth": { 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 5, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "num_secrets": 3 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_active": 1, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_unknown": 0, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "num_osd": 3, 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "pool_sum": { 2026-02-21T01:52:56.878 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 52200, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 101, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 1, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 303, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 101, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 101, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:52:56.879 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 18, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 18, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "up": 3, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 3, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 0 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.880 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum": { 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 85044, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2244, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 28, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 82723, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314487756, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322035462144, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2297856, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 1590359, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 29651, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 84708397 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:52:56.881 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "osd_sum_by_class": { 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "ssd": { 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "up_from": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "num_pgs": 3, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "num_osds": 3, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_osds": 3, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "num_per_pool_omap_osds": 3, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "kb": 314572800, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used": 85044, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_data": 2244, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_omap": 28, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "kb_used_meta": 82723, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "kb_avail": 314487756, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "statfs": { 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "total": 322122547200, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "available": 322035462144, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 2297856, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 1590359, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:56.882 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 29651, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 84708397 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "hb_peers": [], 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "snap_trim_queue_len": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_snap_trimming": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_shards_repaired": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "op_queue_age_hist": { 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "histogram": [], 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "upper_bound": 1 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "perf_stat": { 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ms": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ms": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "commit_latency_ns": 0, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "apply_latency_ns": 0 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "alerts": [], 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "network_ping_times": [] 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "pool_stats": [ 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": 1, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg": 1, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "stat_sum": { 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes": 52200, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects": 101, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_clones": 1, 2026-02-21T01:52:56.883 INFO:tasks.workunit.client.0.trial009.stdout: "num_object_copies": 303, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing_on_primary": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_missing": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_degraded": 101, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_misplaced": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_unfound": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_dirty": 101, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_whiteouts": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_read": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_read_kb": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_write": 101, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_write_kb": 101, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_scrub_errors": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_shallow_scrub_errors": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_deep_scrub_errors": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_recovered": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_recovered": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_keys_recovered": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_omap": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_hit_set_archive": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_bytes_hit_set_archive": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_kb": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_kb": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_promote": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_high": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_flush_mode_low": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_some": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_evict_mode_full": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_pinned": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_legacy_snapsets": 0, 2026-02-21T01:52:56.884 INFO:tasks.workunit.client.0.trial009.stdout: "num_large_omap_objects": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_manifest": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_bytes": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "num_omap_keys": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "num_objects_repaired": 0 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "store_stats": { 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "total": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "available": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "internally_reserved": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "allocated": 1241088, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "data_stored": 1241088, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_allocated": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "data_compressed_original": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "omap_allocated": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "internal_metadata": 0 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "log_size": 18, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "ondisk_log_size": 18, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "up": 3, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "acting": 3, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "num_store_stats": 3 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "osd_stats": [ 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 188978561033 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 38654705679 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.885 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "seq": 206158430217 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_state": [ 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "state": "active+undersized+degraded", 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num": 1 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "pool_pg_unavailable_map": [ 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "poolid": "1", 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "pgs": [] 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_pg_by_osd": [ 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2147483647, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 2, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 0, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 1, 2026-02-21T01:52:56.886 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 1, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "osd": 1, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "num_primary_pg": 0, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "num_acting_pg": 0, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "num_up_not_acting_pg": 0 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": [ 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "pool": 1, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "purged_snaps": {} 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: ], 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "servicemap": { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "epoch": 3, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "modified": "2026-02-21T01:52:08.794214+0000", 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "services": { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "osd": { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "daemons": { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "summary": "", 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "0": { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "2": { 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "start_epoch": 0, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "start_stamp": "0.000000", 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "gid": 0, 2026-02-21T01:52:56.887 INFO:tasks.workunit.client.0.trial009.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "metadata": {}, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "task_status": {} 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: }, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_first_committed": 1, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "mgrstat_last_committed": 58, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "logm_first_committed": 1, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "logm_last_committed": 314, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "paxos": { 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "first_committed": 1, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "last_committed": 432, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "last_pn": 0, 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: "accepted_pn": 0 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout: } 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: return 1 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:341: TEST_divergent_ec: echo 'delay recovery' 2026-02-21T01:52:56.888 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:342: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:52:57.040 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:52:57.040 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-21T01:52:23.348323+0000 41'102 49:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T01:56:54.359962+0000 0 0 2026-02-21T01:52:57.041 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:52:57.041 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:52:57.041 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:52:57.050 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:52:57.051 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.0 2026-02-21T01:52:57.051 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-21T01:52:57.051 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-21T01:52:57.051 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:52:57.051 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:52:57.051 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:52:57.052 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.0.asok 2026-02-21T01:52:57.052 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-21T01:52:57.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.0.asok set_recovery_delay 100000 2026-02-21T01:52:57.109 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:52:57.115 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-21T01:52:57.116 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.2 2026-02-21T01:52:57.116 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-21T01:52:57.116 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-21T01:52:57.116 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:52:57.116 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:52:57.116 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:52:57.117 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.2.asok 2026-02-21T01:52:57.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-21T01:52:57.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.2.asok set_recovery_delay 100000 2026-02-21T01:52:57.172 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:52:57.178 INFO:tasks.workunit.client.0.trial009.stdout:revive divergent 1 2026-02-21T01:52:57.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:349: TEST_divergent_ec: echo 'revive divergent 1' 2026-02-21T01:52:57.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:350: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:52:57.393 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:52:57.393 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-21T01:52:23.348323+0000 41'102 49:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T01:56:54.359962+0000 0 0 2026-02-21T01:52:57.393 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:52:57.393 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:52:57.393 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:52:57.402 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:351: TEST_divergent_ec: ceph osd set noup 2026-02-21T01:52:57.730 INFO:tasks.workunit.client.0.trial009.stderr:noup is set 2026-02-21T01:52:57.745 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:352: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:52:57.746 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:52:57.747 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:52:57.748 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:52:57.749 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:52:57.750 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:52:57.771 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:57.764+0000 7fa04fbcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:57.772 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:57.764+0000 7fa04fbcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:57.773 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:57.764+0000 7fa04fbcfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:52:57.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: return 0 2026-02-21T01:52:57.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:353: TEST_divergent_ec: sleep 5 2026-02-21T01:52:58.156 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:58.148+0000 7fa04fbcfa40 -1 Falling back to public interface 2026-02-21T01:52:58.795 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:52:58.788+0000 7fa04fbcfa40 -1 osd.1 28 log_to_monitors true 2026-02-21T01:53:02.962 INFO:tasks.workunit.client.0.trial009.stdout:delay recovery divergent 2026-02-21T01:53:02.963 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:355: TEST_divergent_ec: echo 'delay recovery divergent' 2026-02-21T01:53:02.963 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:356: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:03.165 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:03.165 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-21T01:52:23.348323+0000 41'102 52:148 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T01:56:54.359962+0000 0 0 2026-02-21T01:53:03.165 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:03.165 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:03.166 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:03.175 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: get_asok_path osd.1 2026-02-21T01:53:03.175 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-21T01:53:03.175 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-21T01:53:03.176 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:53:03.176 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:03.176 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:53:03.176 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.20522/ceph-osd.1.asok 2026-02-21T01:53:03.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: CEPH_ARGS= 2026-02-21T01:53:03.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20522/ceph-osd.1.asok set_recovery_delay 100000 2026-02-21T01:53:03.231 INFO:tasks.workunit.client.0.trial009.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-21T01:53:03.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:359: TEST_divergent_ec: ceph osd unset noup 2026-02-21T01:53:03.562 INFO:tasks.workunit.client.0.trial009.stderr:noup is unset 2026-02-21T01:53:03.575 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:361: TEST_divergent_ec: wait_for_osd up 0 2026-02-21T01:53:03.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:03.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:53:03.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:03.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:03.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:03.576 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:03.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:03.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:03.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:53:03.804 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 44 up_thru 48 down_at 43 last_clean_interval [35,41) [v2:127.0.0.1:6800/1283348058,v1:127.0.0.1:6801/1283348058] [v2:127.0.0.1:6802/1283348058,v1:127.0.0.1:6803/1283348058] exists,up bbfde735-2cd5-4c97-af04-f608bbd9a42f 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:362: TEST_divergent_ec: wait_for_osd up 1 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:53:03.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:03.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:03.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:03.806 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:03.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:03.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:03.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:53:04.019 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 54 up_thru 54 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/1808796184,v1:127.0.0.1:6819/1808796184] [v2:127.0.0.1:6820/1808796184,v1:127.0.0.1:6821/1808796184] exists,up 24ff08c5-1b74-4350-ade4-0abd133531ca 2026-02-21T01:53:04.020 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:363: TEST_divergent_ec: wait_for_osd up 2 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:04.021 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:04.022 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:04.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:04.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:53:04.236 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 48 up_thru 0 down_at 47 last_clean_interval [38,41) [v2:127.0.0.1:6810/1812247902,v1:127.0.0.1:6811/1812247902] [v2:127.0.0.1:6812/1812247902,v1:127.0.0.1:6813/1812247902] exists,up 14f76dc4-b360-4da2-ba08-d2303edbd7e6 2026-02-21T01:53:04.236 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:04.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:04.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:04.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:365: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:04.435 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:04.435 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-21T01:52:23.348323+0000 41'102 52:148 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T01:56:54.359962+0000 0 0 2026-02-21T01:53:04.435 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:04.435 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:04.435 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:04.444 INFO:tasks.workunit.client.0.trial009.stdout:wait for peering 2026-02-21T01:53:04.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:366: TEST_divergent_ec: echo 'wait for peering' 2026-02-21T01:53:04.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:367: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:04.665 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:04.666 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 101 0 0 52200 0 0 18 84 18 active+undersized+degraded 2026-02-21T01:52:23.348323+0000 41'102 52:148 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T01:56:54.359962+0000 0 0 2026-02-21T01:53:04.666 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:04.666 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:04.666 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:04.674 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:368: TEST_divergent_ec: rados -p test put foo /tmp/tmp.S3GfTrhjDp 2026-02-21T01:53:04.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:369: TEST_divergent_ec: rm -f /tmp/tmp.S3GfTrhjDp 2026-02-21T01:53:04.706 INFO:tasks.workunit.client.0.trial009.stdout:killing divergent 1 2026-02-21T01:53:04.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:371: TEST_divergent_ec: echo 'killing divergent 1' 2026-02-21T01:53:04.706 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:372: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:04.909 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:04.909 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 peering 2026-02-21T01:53:03.561528+0000 28'104 55:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:53:04.910 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:04.910 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:04.910 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:04.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:373: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.1 2026-02-21T01:53:04.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:53:04.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:53:04.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:53:04.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:53:04.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:53:05.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:53:05.029 INFO:tasks.workunit.client.0.trial009.stdout:reviving divergent 1 2026-02-21T01:53:05.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:375: TEST_divergent_ec: echo 'reviving divergent 1' 2026-02-21T01:53:05.029 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:376: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:05.224 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:05.225 INFO:tasks.workunit.client.0.trial009.stdout:1.0 101 0 0 0 0 51712 0 0 14 90 14 peering 2026-02-21T01:53:03.561528+0000 28'104 55:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 no scrub is scheduled 0 0 2026-02-21T01:53:05.225 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:05.225 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:05.225 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:05.236 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:377: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-21T01:53:05.236 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/divergent-priors 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/divergent-priors/1 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-21T01:53:05.237 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' 2026-02-21T01:53:05.238 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:05.239 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:53:05.240 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:53:05.240 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/divergent-priors/1 2026-02-21T01:53:05.240 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:53:05.241 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:53:05.241 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=2d0b71ce-c189-4e87-91c9-e8f16ff1b1b0 --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20522/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:05.242 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/divergent-priors/1/whoami 2026-02-21T01:53:05.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:53:05.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:53:05.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:53:05.242 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:53:05.260 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:05.252+0000 7f930e054a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:05.262 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:05.252+0000 7f930e054a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:05.263 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:05.256+0000 7f930e054a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:05.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:53:05.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:05.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:53:05.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:05.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:05.463 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:05.463 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:05.464 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:05.464 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:05.464 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:53:05.640 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:05.632+0000 7f930e054a40 -1 Falling back to public interface 2026-02-21T01:53:05.680 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 54 up_thru 54 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/1808796184,v1:127.0.0.1:6819/1808796184] [v2:127.0.0.1:6820/1808796184,v1:127.0.0.1:6821/1808796184] exists,up 24ff08c5-1b74-4350-ade4-0abd133531ca 2026-02-21T01:53:05.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:05.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:05.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:05.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:379: TEST_divergent_ec: sleep 20 2026-02-21T01:53:06.267 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:06.260+0000 7f930e054a40 -1 osd.1 56 log_to_monitors true 2026-02-21T01:53:25.682 INFO:tasks.workunit.client.0.trial009.stdout:allowing recovery 2026-02-21T01:53:25.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:381: TEST_divergent_ec: echo 'allowing recovery' 2026-02-21T01:53:25.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:382: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:25.884 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:25.884 INFO:tasks.workunit.client.0.trial009.stdout:1.0 102 0 0 0 0 52712 0 0 13 90 13 active+clean 2026-02-21T01:53:06.630970+0000 56'103 60:161 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:55.786514+0000 0 0 2026-02-21T01:53:25.885 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:25.885 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:25.885 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:25.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:384: TEST_divergent_ec: for i in $osds 2026-02-21T01:53:25.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:386: TEST_divergent_ec: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-21T01:53:25.968 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:53:25.975 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:384: TEST_divergent_ec: for i in $osds 2026-02-21T01:53:25.975 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:386: TEST_divergent_ec: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-21T01:53:26.059 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:53:26.066 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:384: TEST_divergent_ec: for i in $osds 2026-02-21T01:53:26.066 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:386: TEST_divergent_ec: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-21T01:53:26.146 INFO:tasks.workunit.client.0.trial009.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-21T01:53:26.153 INFO:tasks.workunit.client.0.trial009.stdout:reading divergent objects 2026-02-21T01:53:26.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:389: TEST_divergent_ec: echo 'reading divergent objects' 2026-02-21T01:53:26.154 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: ceph pg dump pgs 2026-02-21T01:53:26.361 INFO:tasks.workunit.client.0.trial009.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-21T01:53:26.361 INFO:tasks.workunit.client.0.trial009.stdout:1.0 102 0 0 0 0 52712 0 0 13 90 13 active+clean 2026-02-21T01:53:06.630970+0000 56'103 60:161 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-21T01:51:12.710698+0000 0'0 2026-02-21T01:51:12.710698+0000 0 0 periodic scrub scheduled @ 2026-02-22T11:27:55.786514+0000 0 0 2026-02-21T01:53:26.361 INFO:tasks.workunit.client.0.trial009.stdout: 2026-02-21T01:53:26.361 INFO:tasks.workunit.client.0.trial009.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-21T01:53:26.361 INFO:tasks.workunit.client.0.trial009.stderr:dumped pgs 2026-02-21T01:53:26.370 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: expr 2 + 2 2026-02-21T01:53:26.371 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: seq 1 4 2026-02-21T01:53:26.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:53:26.372 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_1 td/divergent-priors/existing 2026-02-21T01:53:26.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:53:26.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_2 td/divergent-priors/existing 2026-02-21T01:53:26.423 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:53:26.423 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_3 td/divergent-priors/existing 2026-02-21T01:53:26.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:391: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-21T01:53:26.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:393: TEST_divergent_ec: rados -p test get existing_4 td/divergent-priors/existing 2026-02-21T01:53:26.474 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:395: TEST_divergent_ec: rm -f td/divergent-priors/existing 2026-02-21T01:53:26.476 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:397: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-21T01:53:26.477 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:397: TEST_divergent_ec: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:53:26.481 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:1 entries: 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-21T01:51:26.033531+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:53:26.481 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: keeping 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-21T01:51:26.033531+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [0~18446744073709551615], clean_omap: true, new_object: false 2026-02-21T01:53:26.481 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:53:26.481 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version: 27'99 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-21T01:53:26.481 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 has no more recent entries in log 2026-02-21T01:53:26.481 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version or op type indicates creation, deleting 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:head entries: 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4563.0:1 2026-02-21T01:51:30.888994+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: keeping 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4563.0:1 2026-02-21T01:51:30.888994+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head prior_version: 27'99 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head has no more recent entries in log 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head olog_can_rollback_to: 28'104 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot rollback 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4563.0:1 2026-02-21T01:51:30.888994+0000 0 ObjectCleanRegions clean_offsets: [512~18446744073709551103], clean_omap: true, new_object: false 2026-02-21T01:53:26.482 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot roll back, removing and adding to missing 2026-02-21T01:53:26.483 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4554.0:1 2026-02-21T01:51:29.820323+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false,27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4560.0:1 2026-02-21T01:51:30.816364+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:53:26.483 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: keeping 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4554.0:1 2026-02-21T01:51:29.820323+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 20 _merge_object_divergent_entries: keeping 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4560.0:1 2026-02-21T01:51:30.816364+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 27'100 first_divergent_update: 27'101 last_divergent_update: 27'102 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 28'104 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4560.0:1 2026-02-21T01:51:30.816364+0000 0 ObjectCleanRegions clean_offsets: [1000~18446744073709550615], clean_omap: true, new_object: false 2026-02-21T01:53:26.484 INFO:tasks.workunit.client.0.trial009.stdout:td/divergent-priors/osd.1.log:2026-02-21T01:53:04.216+0000 7fa02d34c640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-21T01:53:26.485 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:406: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-21T01:53:26.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:406: TEST_divergent_ec: grep -q '_merge_object_divergent_entries.*rolled back' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-21T01:53:26.488 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:411: TEST_divergent_ec: echo success 2026-02-21T01:53:26.489 INFO:tasks.workunit.client.0.trial009.stdout:success 2026-02-21T01:53:26.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:413: TEST_divergent_ec: delete_pool test 2026-02-21T01:53:26.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=test 2026-02-21T01:53:26.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-21T01:53:26.796 INFO:tasks.workunit.client.0.trial009.stderr:pool 'test' does not exist 2026-02-21T01:53:26.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:414: TEST_divergent_ec: kill_daemons td/divergent-priors 2026-02-21T01:53:26.806 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:53:26.806 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:53:26.807 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:53:26.807 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:53:26.807 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:53:32.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:53:32.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-21T01:53:32.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:53:32.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:53:32.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:53:32.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:53:32.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:53:32.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:53:32.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:53:32.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:53:32.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:53:32.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:53:32.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:53:32.133 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:53:32.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:53:32.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:53:32.135 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:53:32.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:53:32.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:53:32.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:53:32.137 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:53:32.138 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:53:32.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:53:32.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:53:32.159 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:53:32.159 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.159 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:53:32.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:53:32.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/divergent-priors 0 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/divergent-priors 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/divergent-priors KILL 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:53:32.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:53:32.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:53:32.164 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:53:32.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:53:32.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:53:32.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:53:32.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:53:32.167 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:53:32.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:53:32.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:53:32.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:53:32.169 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:53:32.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:53:32.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T01:53:32.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/divergent-priors 2026-02-21T01:53:32.173 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:53:32.173 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.173 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.20522 2026-02-21T01:53:32.173 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.20522 2026-02-21T01:53:32.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:53:32.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:53:32.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-21T01:53:32.175 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T01:53:32.176 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T01:53:32.230 INFO:tasks.workunit:Running workunit osd/ec-error-rollforward.sh... 2026-02-21T01:53:32.230 DEBUG:teuthology.orchestra.run.trial009:workunit test osd/ec-error-rollforward.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh 2026-02-21T01:53:32.283 INFO:tasks.workunit.client.0.trial009.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T01:53:32.286 INFO:tasks.workunit.client.0.trial009.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/ec-error-rollforward 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:6: run: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:7: run: shift 2026-02-21T01:53:32.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: export CEPH_MON=127.0.0.1:7132 2026-02-21T01:53:32.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: CEPH_MON=127.0.0.1:7132 2026-02-21T01:53:32.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:11: run: export CEPH_ARGS 2026-02-21T01:53:32.288 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: uuidgen 2026-02-21T01:53:32.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: CEPH_ARGS+='--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none ' 2026-02-21T01:53:32.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:13: run: CEPH_ARGS+='--mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:32.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: export margin=10 2026-02-21T01:53:32.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: margin=10 2026-02-21T01:53:32.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: export objects=200 2026-02-21T01:53:32.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: objects=200 2026-02-21T01:53:32.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: export poolname=test 2026-02-21T01:53:32.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: poolname=test 2026-02-21T01:53:32.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: set 2026-02-21T01:53:32.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T01:53:32.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: local funcs=TEST_ec_error_rollforward 2026-02-21T01:53:32.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:19: run: for func in $funcs 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:20: run: setup td/ec-error-rollforward 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/ec-error-rollforward 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:53:32.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:53:32.293 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:53:32.293 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:53:32.294 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:53:32.294 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:53:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:53:32.295 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:53:32.296 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:53:32.297 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:53:32.297 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:53:32.297 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:53:32.297 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:53:32.298 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:53:32.299 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:53:32.300 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ec-error-rollforward 2026-02-21T01:53:32.301 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:53:32.301 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.301 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.301 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76129 2026-02-21T01:53:32.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:53:32.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:53:32.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/ec-error-rollforward 2026-02-21T01:53:32.304 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:53:32.304 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.304 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.305 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.76129 2026-02-21T01:53:32.306 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:53:32.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T01:53:32.307 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/ec-error-rollforward 1' TERM HUP INT 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:21: run: TEST_ec_error_rollforward td/ec-error-rollforward 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:27: TEST_ec_error_rollforward: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:28: TEST_ec_error_rollforward: run_mon td/ec-error-rollforward a 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/ec-error-rollforward/a 2026-02-21T01:53:32.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/ec-error-rollforward/a --run-dir=td/ec-error-rollforward 2026-02-21T01:53:32.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:53:32.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:32.328 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:32.329 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:32.329 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.329 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.329 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:32.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ec-error-rollforward/a '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --mon-cluster-log-file=td/ec-error-rollforward/log --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:53:32.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:53:32.358 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:53:32.358 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:53:32.358 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:53:32.358 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:53:32.359 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:53:32.359 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:53:32.359 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:53:32.359 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:53:32.359 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:53:32.360 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.360 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.360 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76129/ceph-mon.a.asok 2026-02-21T01:53:32.360 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:53:32.360 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76129/ceph-mon.a.asok config get fsid 2026-02-21T01:53:32.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:53:32.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:53:32.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:53:32.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:53:32.416 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.417 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.76129/ceph-mon.a.asok 2026-02-21T01:53:32.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:53:32.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.76129/ceph-mon.a.asok config get mon_host 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:29: TEST_ec_error_rollforward: run_mgr td/ec-error-rollforward x 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/ec-error-rollforward/x 2026-02-21T01:53:32.468 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:53:32.579 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:53:32.579 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:32.579 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:32.580 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:32.580 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.580 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.580 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:32.580 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:53:32.581 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ec-error-rollforward/x '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:53:32.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:30: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 0 2026-02-21T01:53:32.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:32.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:53:32.598 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/0 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:32.599 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:32.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:32.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:32.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:53:32.600 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/0 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5725c193-ab80-4957-b346-ca30fbcfdfdf 2026-02-21T01:53:32.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 5725c193-ab80-4957-b346-ca30fbcfdfdf' 2026-02-21T01:53:32.602 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 5725c193-ab80-4957-b346-ca30fbcfdfdf 2026-02-21T01:53:32.602 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:53:32.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAcEJlpP+41JBAA1lzvDYIQe1230/hmszUk5w== 2026-02-21T01:53:32.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAcEJlpP+41JBAA1lzvDYIQe1230/hmszUk5w=="}' 2026-02-21T01:53:32.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5725c193-ab80-4957-b346-ca30fbcfdfdf -i td/ec-error-rollforward/0/new.json 2026-02-21T01:53:32.722 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:32.730 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/0/new.json 2026-02-21T01:53:32.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAcEJlpP+41JBAA1lzvDYIQe1230/hmszUk5w== --osd-uuid 5725c193-ab80-4957-b346-ca30fbcfdfdf 2026-02-21T01:53:32.746 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:32.736+0000 7ff83e38ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:32.747 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:32.740+0000 7ff83e38ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:32.748 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:32.740+0000 7ff83e38ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:32.749 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:32.740+0000 7ff83e38ca40 -1 bdev(0x5614cc826c00 td/ec-error-rollforward/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:53:32.749 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:32.740+0000 7ff83e38ca40 -1 bluestore(td/ec-error-rollforward/0) _read_fsid unparsable uuid 2026-02-21T01:53:33.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/0/keyring 2026-02-21T01:53:33.485 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:53:33.486 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:53:33.487 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:53:33.487 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:53:33.604 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:53:33.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:53:33.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:33.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:53:33.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:53:33.605 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:53:33.627 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:33.620+0000 7f8501a7ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:33.629 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:33.620+0000 7f8501a7ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:33.630 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:33.624+0000 7f8501a7ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:33.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:53:33.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:33.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:53:33.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:33.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:33.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:33.711 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:33.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:33.711 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:33.712 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:53:33.816 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:34.067 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:34.060+0000 7f8501a7ba40 -1 Falling back to public interface 2026-02-21T01:53:34.794 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:34.787+0000 7f8501a7ba40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:53:34.817 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:34.817 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:34.817 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:53:34.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:53:34.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:34.818 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:53:34.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:35.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:35.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:35.923 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:53:35.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:53:35.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:35.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:53:36.136 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3724042413,v1:127.0.0.1:6801/3724042413] [v2:127.0.0.1:6802/3724042413,v1:127.0.0.1:6803/3724042413] exists,up 5725c193-ab80-4957-b346-ca30fbcfdfdf 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:31: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 1 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:53:36.140 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/1 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:36.141 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:36.142 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:36.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:36.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:53:36.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:53:36.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/1 2026-02-21T01:53:36.143 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:53:36.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=539639ec-b5c4-4cdf-b866-1c00732ef66c 2026-02-21T01:53:36.144 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 539639ec-b5c4-4cdf-b866-1c00732ef66c 2026-02-21T01:53:36.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 539639ec-b5c4-4cdf-b866-1c00732ef66c' 2026-02-21T01:53:36.144 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:53:36.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgEJlpUT7RCBAAxpogokKnQLmc6x+PcHyJ2A== 2026-02-21T01:53:36.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgEJlpUT7RCBAAxpogokKnQLmc6x+PcHyJ2A=="}' 2026-02-21T01:53:36.152 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 539639ec-b5c4-4cdf-b866-1c00732ef66c -i td/ec-error-rollforward/1/new.json 2026-02-21T01:53:36.367 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:53:36.378 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/1/new.json 2026-02-21T01:53:36.379 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgEJlpUT7RCBAAxpogokKnQLmc6x+PcHyJ2A== --osd-uuid 539639ec-b5c4-4cdf-b866-1c00732ef66c 2026-02-21T01:53:36.403 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:36.395+0000 7f8973abfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:36.405 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:36.395+0000 7f8973abfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:36.406 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:36.399+0000 7f8973abfa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:36.406 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:36.399+0000 7f8973abfa40 -1 bdev(0x56404d69ac00 td/ec-error-rollforward/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:53:36.407 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:36.399+0000 7f8973abfa40 -1 bluestore(td/ec-error-rollforward/1) _read_fsid unparsable uuid 2026-02-21T01:53:37.069 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/1/keyring 2026-02-21T01:53:37.070 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:53:37.071 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:53:37.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:53:37.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:53:37.361 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:53:37.362 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:53:37.362 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:37.363 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:53:37.363 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:53:37.363 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:53:37.385 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:37.375+0000 7f3018f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:37.387 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:37.379+0000 7f3018f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:37.388 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:37.379+0000 7f3018f97a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:37.568 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:37.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:37.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:37.569 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:53:37.731 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:37.723+0000 7f3018f97a40 -1 Falling back to public interface 2026-02-21T01:53:37.775 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:38.410 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:38.403+0000 7f3018f97a40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:53:38.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:38.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:38.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:53:38.777 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:53:38.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:38.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:53:38.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:39.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:39.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:39.994 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:53:39.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:53:39.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:39.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:53:40.207 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4241856167,v1:127.0.0.1:6811/4241856167] [v2:127.0.0.1:6812/4241856167,v1:127.0.0.1:6813/4241856167] exists,up 539639ec-b5c4-4cdf-b866-1c00732ef66c 2026-02-21T01:53:40.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:40.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:40.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:40.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:32: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 2 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/2 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-21T01:53:40.211 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:53:40.212 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:53:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/2 2026-02-21T01:53:40.214 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:53:40.214 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=75a45aee-0d5b-4714-824e-f05080cc3369 2026-02-21T01:53:40.214 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 75a45aee-0d5b-4714-824e-f05080cc3369 2026-02-21T01:53:40.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 75a45aee-0d5b-4714-824e-f05080cc3369' 2026-02-21T01:53:40.215 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:53:40.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAkEJlpcxIdDRAAj2WiNz0Ni0JcmiG3Djhg0A== 2026-02-21T01:53:40.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAkEJlpcxIdDRAAj2WiNz0Ni0JcmiG3Djhg0A=="}' 2026-02-21T01:53:40.224 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 75a45aee-0d5b-4714-824e-f05080cc3369 -i td/ec-error-rollforward/2/new.json 2026-02-21T01:53:40.449 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:53:40.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/2/new.json 2026-02-21T01:53:40.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAkEJlpcxIdDRAAj2WiNz0Ni0JcmiG3Djhg0A== --osd-uuid 75a45aee-0d5b-4714-824e-f05080cc3369 2026-02-21T01:53:40.475 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:40.467+0000 7f5be8f4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:40.477 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:40.467+0000 7f5be8f4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:40.478 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:40.471+0000 7f5be8f4fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:40.478 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:40.471+0000 7f5be8f4fa40 -1 bdev(0x565091c6ac00 td/ec-error-rollforward/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:53:40.479 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:40.471+0000 7f5be8f4fa40 -1 bluestore(td/ec-error-rollforward/2) _read_fsid unparsable uuid 2026-02-21T01:53:41.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/2/keyring 2026-02-21T01:53:41.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:53:41.113 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:53:41.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:53:41.114 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:53:41.408 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:53:41.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:53:41.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:41.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:53:41.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:53:41.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:53:41.430 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:41.423+0000 7f18ae3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:41.432 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:41.423+0000 7f18ae3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:41.433 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:41.423+0000 7f18ae3d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:41.615 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:53:41.615 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:41.615 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:53:41.615 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:41.615 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:41.615 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:41.616 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:41.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:41.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:41.616 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:53:41.787 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:41.779+0000 7f18ae3d6a40 -1 Falling back to public interface 2026-02-21T01:53:41.827 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:42.384 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:42.375+0000 7f18ae3d6a40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:53:42.828 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:42.829 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:42.829 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:53:42.829 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:53:42.830 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:42.830 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:53:43.029 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1896668867,v1:127.0.0.1:6819/1896668867] [v2:127.0.0.1:6820/1896668867,v1:127.0.0.1:6821/1896668867] exists,up 75a45aee-0d5b-4714-824e-f05080cc3369 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:33: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 3 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:53:43.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/ec-error-rollforward/3 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:43.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:43.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:43.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:43.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:53:43.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:53:43.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/ec-error-rollforward/3 2026-02-21T01:53:43.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:53:43.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=33331cb7-7066-43c8-9f48-c242bb8b2474 2026-02-21T01:53:43.037 INFO:tasks.workunit.client.0.trial009.stdout:add osd3 33331cb7-7066-43c8-9f48-c242bb8b2474 2026-02-21T01:53:43.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 33331cb7-7066-43c8-9f48-c242bb8b2474' 2026-02-21T01:53:43.037 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:53:43.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAnEJlpbhOUAhAAZ8hCI/svxlGf/xIWwU/t3Q== 2026-02-21T01:53:43.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAnEJlpbhOUAhAAZ8hCI/svxlGf/xIWwU/t3Q=="}' 2026-02-21T01:53:43.047 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 33331cb7-7066-43c8-9f48-c242bb8b2474 -i td/ec-error-rollforward/3/new.json 2026-02-21T01:53:43.279 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T01:53:43.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/ec-error-rollforward/3/new.json 2026-02-21T01:53:43.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAnEJlpbhOUAhAAZ8hCI/svxlGf/xIWwU/t3Q== --osd-uuid 33331cb7-7066-43c8-9f48-c242bb8b2474 2026-02-21T01:53:43.304 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:43.295+0000 7fda13927a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:43.305 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:43.295+0000 7fda13927a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:43.306 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:43.299+0000 7fda13927a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:43.307 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:43.299+0000 7fda13927a40 -1 bdev(0x555e73d06c00 td/ec-error-rollforward/3/block) open stat got: (1) Operation not permitted 2026-02-21T01:53:43.307 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:43.299+0000 7fda13927a40 -1 bluestore(td/ec-error-rollforward/3) _read_fsid unparsable uuid 2026-02-21T01:53:44.002 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/ec-error-rollforward/3/keyring 2026-02-21T01:53:44.002 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:53:44.003 INFO:tasks.workunit.client.0.trial009.stdout:adding osd3 key to auth repository 2026-02-21T01:53:44.003 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T01:53:44.004 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/ec-error-rollforward/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:53:44.282 INFO:tasks.workunit.client.0.trial009.stdout:start osd.3 2026-02-21T01:53:44.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T01:53:44.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:44.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:53:44.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:53:44.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:53:44.305 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:44.295+0000 7f02ad39da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:44.307 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:44.299+0000 7f02ad39da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:44.308 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:44.299+0000 7f02ad39da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:44.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T01:53:44.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:44.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T01:53:44.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:44.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:44.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:44.499 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:44.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:44.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:44.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:53:44.704 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:44.732 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:44.723+0000 7f02ad39da40 -1 Falling back to public interface 2026-02-21T01:53:45.381 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:45.371+0000 7f02ad39da40 -1 osd.3 0 log_to_monitors true 2026-02-21T01:53:45.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:45.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:45.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:53:45.705 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:53:45.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:45.705 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:53:45.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:53:46.918 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:53:46.918 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:46.919 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:53:46.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:53:46.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:46.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:53:47.129 INFO:tasks.workunit.client.0.trial009.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1122212098,v1:127.0.0.1:6827/1122212098] [v2:127.0.0.1:6828/1122212098,v1:127.0.0.1:6829/1122212098] exists,up 33331cb7-7066-43c8-9f48-c242bb8b2474 2026-02-21T01:53:47.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:47.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:47.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:47.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:35: TEST_ec_error_rollforward: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-21T01:53:47.380 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:36: TEST_ec_error_rollforward: ceph osd pool create ec 1 1 erasure ec-profile 2026-02-21T01:53:47.742 INFO:tasks.workunit.client.0.trial009.stderr:pool 'ec' already exists 2026-02-21T01:53:47.752 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:38: TEST_ec_error_rollforward: rados -p ec put foo /etc/passwd 2026-02-21T01:53:49.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: cat td/ec-error-rollforward/osd.2.pid 2026-02-21T01:53:49.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: kill -STOP 80413 2026-02-21T01:53:49.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:43: TEST_ec_error_rollforward: pids=82517 2026-02-21T01:53:49.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:44: TEST_ec_error_rollforward: sleep 1 2026-02-21T01:53:49.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:42: TEST_ec_error_rollforward: rados -p ec rm foo 2026-02-21T01:53:50.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:46: TEST_ec_error_rollforward: pids+=' 82538' 2026-02-21T01:53:50.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:45: TEST_ec_error_rollforward: rados -p ec rm a 2026-02-21T01:53:50.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:48: TEST_ec_error_rollforward: pids+=' 82539' 2026-02-21T01:53:50.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:47: TEST_ec_error_rollforward: rados -p ec rm b 2026-02-21T01:53:50.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:50: TEST_ec_error_rollforward: pids+=' 82540' 2026-02-21T01:53:50.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:51: TEST_ec_error_rollforward: sleep 1 2026-02-21T01:53:50.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:49: TEST_ec_error_rollforward: rados -p ec rm c 2026-02-21T01:53:51.342 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:54: TEST_ec_error_rollforward: kill_daemons td/ec-error-rollforward KILL osd 2026-02-21T01:53:51.343 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:53:51.343 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:53:51.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:53:51.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:53:51.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:53:51.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:53:51.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:55: TEST_ec_error_rollforward: kill 82517 82538 82539 82540 2026-02-21T01:53:51.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:56: TEST_ec_error_rollforward: wait 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:58: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 0 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/0 2026-02-21T01:53:51.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:51.459 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:53:51.460 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:53:51.461 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/0 2026-02-21T01:53:51.461 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/0/whoami 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:53:51.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:53:51.477 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.467+0000 7f7c03803a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:51.478 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.471+0000 7f7c03803a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:51.479 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.471+0000 7f7c03803a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:51.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:53:51.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:51.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:53:51.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:51.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:51.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:51.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:51.672 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:51.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:51.673 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:53:51.883 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.875+0000 7f7c03803a40 -1 Falling back to public interface 2026-02-21T01:53:51.888 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/3724042413,v1:127.0.0.1:6801/3724042413] [v2:127.0.0.1:6802/3724042413,v1:127.0.0.1:6803/3724042413] exists,up 5725c193-ab80-4957-b346-ca30fbcfdfdf 2026-02-21T01:53:51.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:51.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:51.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:59: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 1 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/1 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:51.892 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:51.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:53:51.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/1 2026-02-21T01:53:51.895 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:53:51.895 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:53:51.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:51.896 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/1/whoami 2026-02-21T01:53:51.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:53:51.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:53:51.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:53:51.896 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:53:51.912 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.903+0000 7fbc2bc6ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:51.914 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.903+0000 7fbc2bc6ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:51.915 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:51.907+0000 7fbc2bc6ea40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:53:52.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:52.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:53:52.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:52.100 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:52.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:52.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:52.101 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:52.101 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:52.102 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:53:52.313 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 10 up_thru 22 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4241856167,v1:127.0.0.1:6811/4241856167] [v2:127.0.0.1:6812/4241856167,v1:127.0.0.1:6813/4241856167] exists,up 539639ec-b5c4-4cdf-b866-1c00732ef66c 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:60: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 2 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:53:52.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/2 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:52.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:52.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:52.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:52.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:53:52.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:53:52.319 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/2 2026-02-21T01:53:52.319 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:53:52.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:53:52.320 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:52.320 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/2/whoami 2026-02-21T01:53:52.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:53:52.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:53:52.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:53:52.321 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:53:52.339 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.331+0000 7f63190a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.340 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.331+0000 7f63190a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.341 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.331+0000 7f63190a7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.391 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.383+0000 7fbc2bc6ea40 -1 Falling back to public interface 2026-02-21T01:53:52.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:53:52.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:52.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:53:52.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:52.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:52.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:52.529 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:52.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:52.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:52.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:53:52.733 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1896668867,v1:127.0.0.1:6819/1896668867] [v2:127.0.0.1:6820/1896668867,v1:127.0.0.1:6821/1896668867] exists,up 75a45aee-0d5b-4714-824e-f05080cc3369 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:61: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 3 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/ec-error-rollforward 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/ec-error-rollforward/3 2026-02-21T01:53:52.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:53:52.738 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:53:52.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:53:52.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/ec-error-rollforward/3 2026-02-21T01:53:52.740 INFO:tasks.workunit.client.0.trial009.stdout:start osd.3 2026-02-21T01:53:52.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T01:53:52.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=1e1653f7-e5d0-4994-82bb-2fa2b74080d3 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.76129/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:53:52.741 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/ec-error-rollforward/3/whoami 2026-02-21T01:53:52.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T01:53:52.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:53:52.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:53:52.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:53:52.753 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.743+0000 7f91ec52fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.755 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.747+0000 7f91ec52fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.756 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.747+0000 7f91ec52fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:53:52.903 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.895+0000 7f7c03803a40 -1 osd.0 24 log_to_monitors true 2026-02-21T01:53:52.931 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:52.923+0000 7f63190a7a40 -1 Falling back to public interface 2026-02-21T01:53:52.956 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T01:53:52.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:53:52.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T01:53:52.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:53:52.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:53:52.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:53:52.957 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:53:52.958 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:53:52.958 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:53:52.958 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:53:53.160 INFO:tasks.workunit.client.0.trial009.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1122212098,v1:127.0.0.1:6827/1122212098] [v2:127.0.0.1:6828/1122212098,v1:127.0.0.1:6829/1122212098] exists,up 33331cb7-7066-43c8-9f48-c242bb8b2474 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:63: TEST_ec_error_rollforward: wait_for_clean 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:53:53.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:53:53.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:53:53.163 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:53:53.163 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:53:53.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:53:53.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:53:53.163 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:53:53.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:53:53.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:53:53.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:53:53.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:53:53.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:53:53.258 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:53:53.335 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:53.327+0000 7f91ec52fa40 -1 Falling back to public interface 2026-02-21T01:53:53.474 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:53:53.474 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:53:53.474 INFO:tasks.workunit.client.0.trial009.stderr:2 2026-02-21T01:53:53.474 INFO:tasks.workunit.client.0.trial009.stderr:3' 2026-02-21T01:53:53.474 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:53:53.474 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:53:53.475 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:53:53.543 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:53.535+0000 7fbc2bc6ea40 -1 osd.1 24 log_to_monitors true 2026-02-21T01:53:53.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-21T01:53:53.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-21T01:53:53.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-115964116995' 2026-02-21T01:53:53.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:53:53.551 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:53:53.991 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:53.983+0000 7f63190a7a40 -1 osd.2 23 log_to_monitors true 2026-02-21T01:53:54.270 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:53:54.259+0000 7f91ec52fa40 -1 osd.3 24 log_to_monitors true 2026-02-21T01:53:58.633 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986179 2026-02-21T01:53:58.633 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986179 2026-02-21T01:53:58.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-115964116995 1-133143986179' 2026-02-21T01:53:58.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:53:58.634 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:53:58.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855363 2026-02-21T01:53:58.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855363 2026-02-21T01:53:58.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-115964116995 1-133143986179 2-150323855363' 2026-02-21T01:53:58.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:53:58.710 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-21T01:53:58.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789955 2026-02-21T01:53:58.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789955 2026-02-21T01:53:58.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-115964116995 1-133143986179 2-150323855363 3-158913789955' 2026-02-21T01:53:58.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:53:58.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-115964116995 2026-02-21T01:53:58.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:53:58.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:53:58.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-115964116995 2026-02-21T01:53:58.789 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:53:58.790 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116995 2026-02-21T01:53:58.790 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 115964116995' 2026-02-21T01:53:58.791 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 115964116995 2026-02-21T01:53:58.791 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:53:59.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116996 -lt 115964116995 2026-02-21T01:53:59.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:53:59.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-133143986179 2026-02-21T01:53:59.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:53:59.003 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:53:59.004 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-133143986179 2026-02-21T01:53:59.004 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:53:59.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986179 2026-02-21T01:53:59.006 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 133143986179 2026-02-21T01:53:59.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 133143986179' 2026-02-21T01:53:59.006 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:53:59.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986180 -lt 133143986179 2026-02-21T01:53:59.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:53:59.218 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-150323855363 2026-02-21T01:53:59.218 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:53:59.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:53:59.219 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-150323855363 2026-02-21T01:53:59.220 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:53:59.221 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855363 2026-02-21T01:53:59.221 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 150323855363 2026-02-21T01:53:59.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 150323855363' 2026-02-21T01:53:59.222 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:53:59.440 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855363 -lt 150323855363 2026-02-21T01:53:59.441 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:53:59.441 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-158913789955 2026-02-21T01:53:59.441 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:53:59.443 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-21T01:53:59.443 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-158913789955 2026-02-21T01:53:59.443 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:53:59.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789955 2026-02-21T01:53:59.445 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.3 seq 158913789955 2026-02-21T01:53:59.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 158913789955' 2026-02-21T01:53:59.446 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T01:53:59.650 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-21T01:53:59.651 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:53:59.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:53:59.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:53:59.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 1 == 0 2026-02-21T01:53:59.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:53:59.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:53:59.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:53:59.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:53:59.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:53:59.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:53:59.920 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:54:00.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=1 2026-02-21T01:54:00.123 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:54:00.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:54:00.124 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:54:00.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 = 1 2026-02-21T01:54:00.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:54:00.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:54:00.408 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:22: run: teardown td/ec-error-rollforward 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ec-error-rollforward 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:54:00.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:54:00.410 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:54:00.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:54:00.518 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:54:00.519 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:54:00.520 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:54:00.521 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:54:00.521 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:54:00.521 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:54:00.522 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:00.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:54:00.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:54:00.523 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:00.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:54:00.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:54:00.526 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ec-error-rollforward 2026-02-21T01:54:00.538 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:54:00.538 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.538 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:54:00.538 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76129 2026-02-21T01:54:00.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:54:00.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:54:00.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-21T01:54:00.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/ec-error-rollforward 0 2026-02-21T01:54:00.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/ec-error-rollforward 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:54:00.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:54:00.542 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:54:00.543 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:54:00.543 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:54:00.544 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:54:00.545 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:54:00.545 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:54:00.545 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:54:00.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:00.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:54:00.546 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:54:00.546 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:00.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:54:00.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T01:54:00.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/ec-error-rollforward 2026-02-21T01:54:00.549 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:54:00.550 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.550 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.76129 2026-02-21T01:54:00.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.76129 2026-02-21T01:54:00.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:54:00.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:54:00.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-21T01:54:00.552 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T01:54:00.552 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T01:54:00.606 INFO:tasks.workunit:Running workunit osd/osd-bench.sh... 2026-02-21T01:54:00.606 DEBUG:teuthology.orchestra.run.trial009:workunit test osd/osd-bench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh 2026-02-21T01:54:00.659 INFO:tasks.workunit.client.0.trial009.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T01:54:00.662 INFO:tasks.workunit.client.0.trial009.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T01:54:00.662 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:54:00.662 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:54:00.662 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-21T01:54:00.662 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-21T01:54:00.662 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-bench 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:22: run: local dir=td/osd-bench 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:23: run: shift 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: export CEPH_MON=127.0.0.1:7106 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: CEPH_MON=127.0.0.1:7106 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:26: run: export CEPH_ARGS 2026-02-21T01:54:00.663 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: uuidgen 2026-02-21T01:54:00.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: CEPH_ARGS+='--fsid=6aab4605-985b-41f4-b198-091f809bbc0e --auth-supported=none ' 2026-02-21T01:54:00.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7106 ' 2026-02-21T01:54:00.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:29: run: CEPH_ARGS+='--debug-bluestore 20 ' 2026-02-21T01:54:00.664 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: set 2026-02-21T01:54:00.665 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T01:54:00.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: local funcs=TEST_bench 2026-02-21T01:54:00.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:32: run: for func in $funcs 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:33: run: setup td/osd-bench 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bench 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bench 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bench 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bench KILL 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:54:00.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:54:00.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:54:00.668 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:54:00.669 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:54:00.669 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:54:00.670 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:54:00.670 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:54:00.670 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:54:00.671 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:00.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:54:00.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:54:00.672 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:00.673 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:54:00.674 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:54:00.674 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bench 2026-02-21T01:54:00.676 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:54:00.676 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.676 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.676 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85123 2026-02-21T01:54:00.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:54:00.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:54:00.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bench 2026-02-21T01:54:00.679 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:54:00.679 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.679 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.85123 2026-02-21T01:54:00.681 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:54:00.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T01:54:00.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T01:54:00.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:54:00.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bench 1' TERM HUP INT 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:34: run: TEST_bench td/osd-bench 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:40: TEST_bench: local dir=td/osd-bench 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:42: TEST_bench: run_mon td/osd-bench a 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bench 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bench/a 2026-02-21T01:54:00.683 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bench/a --run-dir=td/osd-bench 2026-02-21T01:54:00.707 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:54:00.707 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:00.707 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:00.707 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:00.707 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.707 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.708 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85123/$cluster-$name.asok' 2026-02-21T01:54:00.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bench/a '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.85123/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bench/log --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:54:00.732 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:54:00.735 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:54:00.735 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:54:00.735 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:54:00.735 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:54:00.735 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:54:00.735 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85123/ceph-mon.a.asok 2026-02-21T01:54:00.736 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:54:00.737 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85123/ceph-mon.a.asok config get fsid 2026-02-21T01:54:00.790 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:54:00.790 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:54:00.790 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:54:00.790 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:54:00.790 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.85123/ceph-mon.a.asok 2026-02-21T01:54:00.791 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:54:00.792 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.85123/ceph-mon.a.asok config get mon_host 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:43: TEST_bench: run_mgr td/osd-bench x 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bench 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bench/x 2026-02-21T01:54:00.847 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:54:00.960 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:54:00.960 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:00.960 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:00.960 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:00.960 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.960 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.961 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85123/$cluster-$name.asok' 2026-02-21T01:54:00.961 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:54:00.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bench/x '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.85123/$cluster-$name.asok' --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:44: TEST_bench: run_osd td/osd-bench 0 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bench 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bench/0 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6aab4605-985b-41f4-b198-091f809bbc0e --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 ' 2026-02-21T01:54:00.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bench/0' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bench/0/journal' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bench' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:00.983 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.85123/$cluster-$name.asok' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.85123/$cluster-$name.asok' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bench/$name.log' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bench/$name.pid' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:54:00.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:54:00.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:54:00.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:54:00.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bench/0 2026-02-21T01:54:00.985 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:54:00.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=22115f7d-16d1-4424-afe8-1f0c6e0d355c 2026-02-21T01:54:00.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 22115f7d-16d1-4424-afe8-1f0c6e0d355c' 2026-02-21T01:54:00.986 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 22115f7d-16d1-4424-afe8-1f0c6e0d355c 2026-02-21T01:54:00.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:54:00.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA4EJlpyKYGOxAA9Y4wJSmeMEwVv3le8jwolA== 2026-02-21T01:54:00.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA4EJlpyKYGOxAA9Y4wJSmeMEwVv3le8jwolA=="}' 2026-02-21T01:54:00.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 22115f7d-16d1-4424-afe8-1f0c6e0d355c -i td/osd-bench/0/new.json 2026-02-21T01:54:01.101 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:01.109 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bench/0/new.json 2026-02-21T01:54:01.110 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6aab4605-985b-41f4-b198-091f809bbc0e --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.85123/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA4EJlpyKYGOxAA9Y4wJSmeMEwVv3le8jwolA== --osd-uuid 22115f7d-16d1-4424-afe8-1f0c6e0d355c 2026-02-21T01:54:01.126 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:01.115+0000 7f437b9a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:01.127 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:01.119+0000 7f437b9a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:01.128 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:01.119+0000 7f437b9a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:01.129 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:01.119+0000 7f437b9a4a40 -1 bdev(0x55c317326c00 td/osd-bench/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:54:01.129 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:01.119+0000 7f437b9a4a40 -1 bluestore(td/osd-bench/0) _read_fsid unparsable uuid 2026-02-21T01:54:01.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bench/0/keyring 2026-02-21T01:54:01.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:54:01.934 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:54:01.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:54:01.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bench/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:54:02.056 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:54:02.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:54:02.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6aab4605-985b-41f4-b198-091f809bbc0e --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.85123/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:54:02.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:54:02.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:54:02.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:54:02.078 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:02.067+0000 7fb17e1eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:02.080 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:02.071+0000 7fb17e1eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:02.081 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:02.071+0000 7fb17e1eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:02.168 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:54:02.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:54:02.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:54:02.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:54:02.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:54:02.169 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:02.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:02.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:54:02.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:02.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:54:02.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:02.496 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:02.487+0000 7fb17e1eaa40 -1 Falling back to public interface 2026-02-21T01:54:03.155 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:03.147+0000 7fb17e1eaa40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:54:03.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:03.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:03.280 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:54:03.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:54:03.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:03.281 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:54:03.520 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:04.522 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:04.522 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:04.522 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:54:04.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:54:04.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:54:04.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:04.728 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/1333301028,v1:127.0.0.1:6801/1333301028] [v2:127.0.0.1:6802/1333301028,v1:127.0.0.1:6803/1333301028] exists,up 22115f7d-16d1-4424-afe8-1f0c6e0d355c 2026-02-21T01:54:04.729 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:54:04.729 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:54:04.730 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:54:04.730 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: CEPH_ARGS= 2026-02-21T01:54:04.730 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: ceph-conf --show-config-value osd_bench_small_size_max_iops 2026-02-21T01:54:04.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: local osd_bench_small_size_max_iops=100 2026-02-21T01:54:04.739 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: CEPH_ARGS= 2026-02-21T01:54:04.739 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: ceph-conf --show-config-value osd_bench_large_size_max_throughput 2026-02-21T01:54:04.753 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: local osd_bench_large_size_max_throughput=104857600 2026-02-21T01:54:04.753 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: CEPH_ARGS= 2026-02-21T01:54:04.753 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: ceph-conf --show-config-value osd_bench_max_block_size 2026-02-21T01:54:04.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: local osd_bench_max_block_size=67108864 2026-02-21T01:54:04.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: CEPH_ARGS= 2026-02-21T01:54:04.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: ceph-conf --show-config-value osd_bench_duration 2026-02-21T01:54:04.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: local osd_bench_duration=30 2026-02-21T01:54:04.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:58: TEST_bench: expect_failure td/osd-bench osd_bench_max_block_size ceph tell osd.0 bench 1024 67108865 2026-02-21T01:54:04.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/osd-bench 2026-02-21T01:54:04.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-21T01:54:04.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=osd_bench_max_block_size 2026-02-21T01:54:04.776 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-21T01:54:04.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-21T01:54:04.777 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: ceph tell osd.0 bench 1024 67108865 2026-02-21T01:54:04.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-21T01:54:04.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-21T01:54:04.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet osd_bench_max_block_size td/osd-bench/out 2026-02-21T01:54:04.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-21T01:54:04.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:64: TEST_bench: local bsize=1024 2026-02-21T01:54:04.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:65: TEST_bench: local max_count=3072000 2026-02-21T01:54:04.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:66: TEST_bench: expect_failure td/osd-bench bench_small_size_max_iops ceph tell osd.0 bench 3072001 1024 2026-02-21T01:54:04.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/osd-bench 2026-02-21T01:54:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-21T01:54:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=bench_small_size_max_iops 2026-02-21T01:54:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-21T01:54:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-21T01:54:04.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: ceph tell osd.0 bench 3072001 1024 2026-02-21T01:54:04.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-21T01:54:04.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-21T01:54:04.921 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet bench_small_size_max_iops td/osd-bench/out 2026-02-21T01:54:04.922 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:72: TEST_bench: local bsize=1048577 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:73: TEST_bench: local max_count=3145728000 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:74: TEST_bench: expect_failure td/osd-bench osd_bench_large_size_max_throughput ceph tell osd.0 bench 3145728001 1048577 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2016: expect_failure: local dir=td/osd-bench 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2017: expect_failure: shift 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2018: expect_failure: local expected=osd_bench_large_size_max_throughput 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2019: expect_failure: shift 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2020: expect_failure: local success 2026-02-21T01:54:04.923 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2022: expect_failure: ceph tell osd.0 bench 3145728001 1048577 2026-02-21T01:54:04.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: success=false 2026-02-21T01:54:04.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: false 2026-02-21T01:54:04.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: grep --quiet osd_bench_large_size_max_throughput td/osd-bench/out 2026-02-21T01:54:04.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2032: expect_failure: return 0 2026-02-21T01:54:04.996 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:80: TEST_bench: ceph tell osd.0 bench 2026-02-21T01:54:06.660 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:54:06.660 INFO:tasks.workunit.client.0.trial009.stdout: "bytes_written": 1073741824, 2026-02-21T01:54:06.661 INFO:tasks.workunit.client.0.trial009.stdout: "blocksize": 4194304, 2026-02-21T01:54:06.661 INFO:tasks.workunit.client.0.trial009.stdout: "elapsed_sec": 0.74653109200000001, 2026-02-21T01:54:06.661 INFO:tasks.workunit.client.0.trial009.stdout: "bytes_per_sec": 1438308243.965276, 2026-02-21T01:54:06.661 INFO:tasks.workunit.client.0.trial009.stdout: "iops": 342.91940783626461 2026-02-21T01:54:06.661 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:54:06.675 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:84: TEST_bench: ceph tell osd.0 bench 10 14456 4444 3 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout: "bytes_written": 10, 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout: "blocksize": 14456, 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout: "elapsed_sec": 0.00037548799999999999, 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout: "bytes_per_sec": 26632.00954491222, 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout: "iops": 1.8422806824095337 2026-02-21T01:54:06.745 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:54:06.752 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:90: TEST_bench: ceph tell osd.0 bench 1 14456 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout:{ 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout: "bytes_written": 1, 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout: "blocksize": 14456, 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout: "elapsed_sec": 0.00024875199999999998, 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout: "bytes_per_sec": 4020.0681803563393, 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout: "iops": 0.27808994053378105 2026-02-21T01:54:06.830 INFO:tasks.workunit.client.0.trial009.stdout:} 2026-02-21T01:54:06.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:35: run: teardown td/osd-bench 2026-02-21T01:54:06.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bench 2026-02-21T01:54:06.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:54:06.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bench KILL 2026-02-21T01:54:06.838 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:54:06.838 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:54:06.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:54:06.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:54:06.839 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:54:06.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:54:06.947 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:54:06.948 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:54:06.948 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:54:06.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:54:06.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:54:06.950 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:54:06.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:06.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:54:06.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:54:06.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:06.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:54:06.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:54:06.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bench 2026-02-21T01:54:06.987 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:54:06.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:06.988 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:06.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85123 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-bench 0 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bench 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bench KILL 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:54:06.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:54:06.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:54:06.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:54:06.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:54:06.992 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:54:06.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:54:06.993 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:54:06.994 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:54:06.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:54:06.995 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:54:06.996 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:06.996 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:54:06.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:54:06.997 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:06.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:54:06.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-21T01:54:06.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bench 2026-02-21T01:54:07.000 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:54:07.000 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.85123 2026-02-21T01:54:07.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.85123 2026-02-21T01:54:07.002 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:54:07.002 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:54:07.002 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-21T01:54:07.003 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T01:54:07.003 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T01:54:07.054 INFO:tasks.workunit:Running workunit osd/osd-bluefs-volume-ops.sh... 2026-02-21T01:54:07.054 DEBUG:teuthology.orchestra.run.trial009:workunit test osd/osd-bluefs-volume-ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh 2026-02-21T01:54:07.107 INFO:tasks.workunit.client.0.trial009.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-21T01:54:07.112 INFO:tasks.workunit.client.0.trial009.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/osd-bluefs-volume-ops 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:8: run: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.113 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:9: run: shift 2026-02-21T01:54:07.114 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: set 2026-02-21T01:54:07.114 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: local 'funcs=TEST_bluestore 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:TEST_bluestore2 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:TEST_bluestore_expand' 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bluefs-volume-ops 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.116 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T01:54:07.117 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T01:54:07.118 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T01:54:07.119 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T01:54:07.119 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T01:54:07.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T01:54:07.120 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T01:54:07.121 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T01:54:07.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:07.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T01:54:07.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T01:54:07.122 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T01:54:07.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T01:54:07.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T01:54:07.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-21T01:54:07.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T01:54:07.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.126 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.87226 2026-02-21T01:54:07.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T01:54:07.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T01:54:07.128 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-21T01:54:07.129 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T01:54:07.129 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.130 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.87226 2026-02-21T01:54:07.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T01:54:07.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-21T01:54:07.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-21T01:54:07.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T01:54:07.132 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-21T01:54:07.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore td/osd-bluefs-volume-ops 2026-02-21T01:54:07.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:20: TEST_bluestore: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.133 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: ulimit -n 2026-02-21T01:54:07.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: local flimit=4096 2026-02-21T01:54:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:23: TEST_bluestore: '[' 4096 -lt 1536 ']' 2026-02-21T01:54:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: export CEPH_MON=127.0.0.1:7146 2026-02-21T01:54:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: CEPH_MON=127.0.0.1:7146 2026-02-21T01:54:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:27: TEST_bluestore: export CEPH_ARGS 2026-02-21T01:54:07.134 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: uuidgen 2026-02-21T01:54:07.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: CEPH_ARGS+='--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:29: TEST_bluestore: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:30: TEST_bluestore: CEPH_ARGS+='--bluestore_block_size=2147483648 ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:31: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:32: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:33: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_size=536870912 ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:34: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_create=true ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:35: TEST_bluestore: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:37: TEST_bluestore: CEPH_ARGS+='--bluestore_allocation_from_file=1 ' 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:39: TEST_bluestore: run_mon td/osd-bluefs-volume-ops a 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T01:54:07.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T01:54:07.137 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T01:54:07.137 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-21T01:54:07.137 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T01:54:07.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:07.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:07.161 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:07.162 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.162 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:07.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T01:54:07.191 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T01:54:07.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T01:54:07.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:54:07.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:54:07.193 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T01:54:07.194 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T01:54:07.194 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:54:07.194 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:54:07.194 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:54:07.195 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:54:07.195 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.195 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.195 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.87226/ceph-mon.a.asok 2026-02-21T01:54:07.195 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:54:07.195 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.87226/ceph-mon.a.asok config get fsid 2026-02-21T01:54:07.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T01:54:07.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T01:54:07.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T01:54:07.252 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T01:54:07.253 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T01:54:07.253 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.87226/ceph-mon.a.asok 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T01:54:07.254 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.87226/ceph-mon.a.asok config get mon_host 2026-02-21T01:54:07.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:40: TEST_bluestore: run_mgr td/osd-bluefs-volume-ops x 2026-02-21T01:54:07.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T01:54:07.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T01:54:07.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T01:54:07.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-21T01:54:07.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T01:54:07.423 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T01:54:07.423 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:07.423 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:07.424 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:07.424 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.424 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.424 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:07.425 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:54:07.425 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T01:54:07.444 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:41: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 0 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:54:07.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:07.446 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T01:54:07.447 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:54:07.448 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T01:54:07.448 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 52e40f1b-ea5c-4ac8-9da8-b96001b82630' 2026-02-21T01:54:07.448 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T01:54:07.449 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:54:07.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQA/EJlpXDcDGxAAFe3yYBQ7+9Vrb47sRy0Tag== 2026-02-21T01:54:07.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQA/EJlpXDcDGxAAFe3yYBQ7+9Vrb47sRy0Tag=="}' 2026-02-21T01:54:07.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 52e40f1b-ea5c-4ac8-9da8-b96001b82630 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-21T01:54:07.564 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:07.571 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-21T01:54:07.573 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA/EJlpXDcDGxAAFe3yYBQ7+9Vrb47sRy0Tag== --osd-uuid 52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T01:54:07.588 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:07.579+0000 7fbd582eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:07.589 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:07.579+0000 7fbd582eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:07.590 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:07.583+0000 7fbd582eaa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:07.591 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:07.583+0000 7fbd582eaa40 -1 bdev(0x565352b82c00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-21T01:54:07.591 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:07.583+0000 7fbd582eaa40 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-21T01:54:09.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-21T01:54:09.429 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:54:09.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T01:54:09.430 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T01:54:09.430 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:54:09.584 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T01:54:09.585 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:54:09.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:54:09.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:54:09.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:54:09.588 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:54:09.602 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:09.595+0000 7f058d22aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:09.604 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:09.595+0000 7f058d22aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:09.605 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:09.595+0000 7f058d22aa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:09.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T01:54:09.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:54:09.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:54:09.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:54:09.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:54:09.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:09.697 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:09.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:54:09.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:09.698 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:54:09.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:10.136 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:10.127+0000 7f058d22aa40 -1 Falling back to public interface 2026-02-21T01:54:10.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:10.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:10.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:54:10.910 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:54:10.910 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:10.910 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:54:11.119 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:11.291 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:11.283+0000 7f058d22aa40 -1 osd.0 0 log_to_monitors true 2026-02-21T01:54:12.121 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:12.121 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:12.121 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:54:12.121 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:54:12.121 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:12.121 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:54:12.326 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3717591817,v1:127.0.0.1:6803/3717591817] [v2:127.0.0.1:6804/3717591817,v1:127.0.0.1:6805/3717591817] exists,up 52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T01:54:12.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:54:12.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:54:12.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:54:12.327 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T01:54:12.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: osd_pid0=88624 2026-02-21T01:54:12.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:43: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 1 2026-02-21T01:54:12.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:12.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:54:12.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:54:12.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:54:12.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-21T01:54:12.334 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:54:12.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T01:54:12.334 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 78fc2c9f-9040-4ea9-9180-d6f41d663f2f' 2026-02-21T01:54:12.334 INFO:tasks.workunit.client.0.trial009.stdout:add osd1 78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T01:54:12.335 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:54:12.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEEJlpA81HFBAATCca6f5JYWCY19oJdkblfQ== 2026-02-21T01:54:12.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEEJlpA81HFBAATCca6f5JYWCY19oJdkblfQ=="}' 2026-02-21T01:54:12.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 78fc2c9f-9040-4ea9-9180-d6f41d663f2f -i td/osd-bluefs-volume-ops/1/new.json 2026-02-21T01:54:12.555 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:54:12.564 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/1/new.json 2026-02-21T01:54:12.565 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEEJlpA81HFBAATCca6f5JYWCY19oJdkblfQ== --osd-uuid 78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T01:54:12.581 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:12.571+0000 7f93a5ca7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:12.582 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:12.575+0000 7f93a5ca7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:12.583 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:12.575+0000 7f93a5ca7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:12.584 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:12.575+0000 7f93a5ca7a40 -1 bdev(0x565443f54c00 td/osd-bluefs-volume-ops/1/block) open stat got: (1) Operation not permitted 2026-02-21T01:54:12.584 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:12.575+0000 7f93a5ca7a40 -1 bluestore(td/osd-bluefs-volume-ops/1) _read_fsid unparsable uuid 2026-02-21T01:54:14.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/1/keyring 2026-02-21T01:54:14.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:54:14.173 INFO:tasks.workunit.client.0.trial009.stdout:adding osd1 key to auth repository 2026-02-21T01:54:14.173 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-21T01:54:14.173 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:54:14.455 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:54:14.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-21T01:54:14.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:54:14.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:54:14.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:54:14.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:54:14.478 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:14.471+0000 7faf1c537a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:14.480 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:14.471+0000 7faf1c537a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:14.481 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:14.471+0000 7faf1c537a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:14.667 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:14.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:54:14.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:14.668 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:54:14.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:14.972 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:14.963+0000 7faf1c537a40 -1 Falling back to public interface 2026-02-21T01:54:15.881 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:15.882 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:15.882 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:54:15.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:54:15.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:15.883 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:54:16.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:16.172 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:16.163+0000 7faf1c537a40 -1 osd.1 0 log_to_monitors true 2026-02-21T01:54:17.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:17.089 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:17.090 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:54:17.090 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:54:17.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:17.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:54:17.300 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1598712006,v1:127.0.0.1:6811/1598712006] [v2:127.0.0.1:6812/1598712006,v1:127.0.0.1:6813/1598712006] exists,up 78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T01:54:17.301 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:54:17.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:54:17.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:54:17.302 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-21T01:54:17.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: osd_pid1=91074 2026-02-21T01:54:17.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:45: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 2 2026-02-21T01:54:17.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:17.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:54:17.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:54:17.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-21T01:54:17.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-21T01:54:17.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:54:17.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:54:17.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:54:17.311 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:54:17.311 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:17.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:54:17.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T01:54:17.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 130cca78-4d57-40fb-a845-e0ee751e63c8' 2026-02-21T01:54:17.315 INFO:tasks.workunit.client.0.trial009.stdout:add osd2 130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T01:54:17.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:54:17.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBJEJlp74G0EhAA+GRVuZoMaoJSNYPkSEXZaQ== 2026-02-21T01:54:17.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBJEJlp74G0EhAA+GRVuZoMaoJSNYPkSEXZaQ=="}' 2026-02-21T01:54:17.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 130cca78-4d57-40fb-a845-e0ee751e63c8 -i td/osd-bluefs-volume-ops/2/new.json 2026-02-21T01:54:17.538 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:54:17.547 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/2/new.json 2026-02-21T01:54:17.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBJEJlp74G0EhAA+GRVuZoMaoJSNYPkSEXZaQ== --osd-uuid 130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T01:54:17.564 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:17.555+0000 7f8689173a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:17.565 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:17.555+0000 7f8689173a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:17.566 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:17.559+0000 7f8689173a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:17.567 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:17.559+0000 7f8689173a40 -1 bdev(0x560a3c9f8c00 td/osd-bluefs-volume-ops/2/block) open stat got: (1) Operation not permitted 2026-02-21T01:54:17.567 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:17.559+0000 7f8689173a40 -1 bluestore(td/osd-bluefs-volume-ops/2) _read_fsid unparsable uuid 2026-02-21T01:54:19.026 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/2/keyring 2026-02-21T01:54:19.026 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:54:19.026 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-21T01:54:19.027 INFO:tasks.workunit.client.0.trial009.stdout:adding osd2 key to auth repository 2026-02-21T01:54:19.027 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:54:19.321 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:54:19.322 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-21T01:54:19.322 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:54:19.322 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:54:19.323 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:54:19.323 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:54:19.345 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:19.335+0000 7f09f467da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:19.347 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:19.339+0000 7f09f467da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:19.348 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:19.339+0000 7f09f467da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:19.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-21T01:54:19.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:54:19.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:54:19.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:54:19.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:54:19.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:19.524 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:19.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:54:19.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:19.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:54:19.737 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:19.832 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:19.823+0000 7f09f467da40 -1 Falling back to public interface 2026-02-21T01:54:20.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:20.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:20.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:54:20.739 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:54:20.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:20.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:54:20.948 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:21.099 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:21.095+0000 7f09f467da40 -1 osd.2 0 log_to_monitors true 2026-02-21T01:54:21.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:21.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:21.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:54:21.951 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:54:21.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:21.951 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:54:22.157 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2900426187,v1:127.0.0.1:6819/2900426187] [v2:127.0.0.1:6820/2900426187,v1:127.0.0.1:6821/2900426187] exists,up 130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T01:54:22.157 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:54:22.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:54:22.158 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:54:22.158 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-21T01:54:22.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: osd_pid2=93374 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:47: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 3 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:54:22.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:54:22.162 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:22.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:54:22.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:54:22.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T01:54:22.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T01:54:22.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T01:54:22.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T01:54:22.166 INFO:tasks.workunit.client.0.trial009.stdout:add osd3 e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T01:54:22.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e1c504e5-b193-4b0a-a81c-ad143b0184ca' 2026-02-21T01:54:22.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T01:54:22.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBOEJlprWhEChAAm6tqsS0uuWfI8ZoyeANTGA== 2026-02-21T01:54:22.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBOEJlprWhEChAAm6tqsS0uuWfI8ZoyeANTGA=="}' 2026-02-21T01:54:22.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e1c504e5-b193-4b0a-a81c-ad143b0184ca -i td/osd-bluefs-volume-ops/3/new.json 2026-02-21T01:54:22.397 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T01:54:22.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/3/new.json 2026-02-21T01:54:22.409 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBOEJlprWhEChAAm6tqsS0uuWfI8ZoyeANTGA== --osd-uuid e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T01:54:22.432 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:22.427+0000 7fa91c370a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:22.434 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:22.431+0000 7fa91c370a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:22.435 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:22.431+0000 7fa91c370a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:22.436 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:22.431+0000 7fa91c370a40 -1 bdev(0x55e72c0cac00 td/osd-bluefs-volume-ops/3/block) open stat got: (1) Operation not permitted 2026-02-21T01:54:22.436 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:22.431+0000 7fa91c370a40 -1 bluestore(td/osd-bluefs-volume-ops/3) _read_fsid unparsable uuid 2026-02-21T01:54:23.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/3/keyring 2026-02-21T01:54:23.837 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T01:54:23.838 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-21T01:54:23.839 INFO:tasks.workunit.client.0.trial009.stdout:adding osd3 key to auth repository 2026-02-21T01:54:23.839 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T01:54:24.117 INFO:tasks.workunit.client.0.trial009.stdout:start osd.3 2026-02-21T01:54:24.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-21T01:54:24.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:54:24.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T01:54:24.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T01:54:24.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T01:54:24.140 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:24.135+0000 7f60169f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:24.142 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:24.139+0000 7f60169f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:24.143 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:24.139+0000 7f60169f9a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:54:24.328 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:54:24.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:24.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:54:24.539 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:24.607 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:24.603+0000 7f60169f9a40 -1 Falling back to public interface 2026-02-21T01:54:25.540 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:25.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:25.541 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:54:25.541 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:54:25.542 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:25.542 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:54:25.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:54:25.844 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:54:25.839+0000 7f60169f9a40 -1 osd.3 0 log_to_monitors true 2026-02-21T01:54:26.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:54:26.744 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:54:26.744 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:54:26.745 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:54:26.745 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:54:26.745 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:54:26.951 INFO:tasks.workunit.client.0.trial009.stdout:osd.3 up in weight 1 up_from 19 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1946158249,v1:127.0.0.1:6827/1946158249] [v2:127.0.0.1:6828/1946158249,v1:127.0.0.1:6829/1946158249] exists,up e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T01:54:26.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:54:26.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:54:26.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:54:26.952 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-21T01:54:26.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: osd_pid3=95584 2026-02-21T01:54:26.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:50: TEST_bluestore: sleep 5 2026-02-21T01:54:31.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:52: TEST_bluestore: create_pool foo 16 2026-02-21T01:54:31.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-21T01:54:32.218 INFO:tasks.workunit.client.0.trial009.stderr:pool 'foo' already exists 2026-02-21T01:54:32.227 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T01:54:33.229 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:55: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-21T01:54:33.251 INFO:tasks.workunit.client.0.trial009.stdout:hints = 1 2026-02-21T01:54:33.251 INFO:tasks.workunit.client.0.trial009.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-21T01:54:33.251 INFO:tasks.workunit.client.0.trial009.stdout:Object prefix: benchmark_data_trial009_96542 2026-02-21T01:54:33.251 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T01:54:33.251 INFO:tasks.workunit.client.0.trial009.stdout: 0 0 0 0 0 0 - 0 2026-02-21T01:54:34.251 INFO:tasks.workunit.client.0.trial009.stdout: 1 16 10988 10972 42.8559 42.8594 0.00101582 0.00145565 2026-02-21T01:54:35.251 INFO:tasks.workunit.client.0.trial009.stdout: 2 16 22016 22000 42.9651 43.0781 0.00141747 0.00145275 2026-02-21T01:54:36.251 INFO:tasks.workunit.client.0.trial009.stdout: 3 16 33073 33057 43.0394 43.1914 0.00108197 0.00145036 2026-02-21T01:54:37.251 INFO:tasks.workunit.client.0.trial009.stdout: 4 16 44045 44029 42.9935 42.8594 0.00153628 0.00145201 2026-02-21T01:54:38.251 INFO:tasks.workunit.client.0.trial009.stdout: 5 15 55144 55129 43.066 43.3594 0.00120409 0.00144959 2026-02-21T01:54:39.251 INFO:tasks.workunit.client.0.trial009.stdout: 6 16 66309 66293 43.1561 43.6094 0.00131675 0.00144659 2026-02-21T01:54:40.251 INFO:tasks.workunit.client.0.trial009.stdout: 7 15 77576 77561 43.2783 44.0156 0.00369215 0.0014425 2026-02-21T01:54:41.251 INFO:tasks.workunit.client.0.trial009.stdout: 8 16 88563 88547 43.2324 42.9141 0.00117729 0.00144413 2026-02-21T01:54:42.251 INFO:tasks.workunit.client.0.trial009.stdout: 9 16 93502 93486 40.5723 19.293 0.00159717 0.00153885 2026-02-21T01:54:43.251 INFO:tasks.workunit.client.0.trial009.stdout: 10 16 104514 104498 40.8163 43.0156 0.00100917 0.0015297 2026-02-21T01:54:44.251 INFO:tasks.workunit.client.0.trial009.stdout: 11 16 115502 115486 41.0074 42.9219 0.00106698 0.00152261 2026-02-21T01:54:45.251 INFO:tasks.workunit.client.0.trial009.stdout: 12 16 126581 126565 41.1963 43.2773 0.00150881 0.00151561 2026-02-21T01:54:46.252 INFO:tasks.workunit.client.0.trial009.stdout: 13 16 137320 137304 41.254 41.9492 0.00149955 0.00151347 2026-02-21T01:54:47.252 INFO:tasks.workunit.client.0.trial009.stdout: 14 16 145204 145188 40.5069 30.7969 0.101101 0.00153143 2026-02-21T01:54:48.252 INFO:tasks.workunit.client.0.trial009.stdout: 15 16 145276 145260 37.8252 0.28125 0.185386 0.00164036 2026-02-21T01:54:49.252 INFO:tasks.workunit.client.0.trial009.stdout: 16 16 145380 145364 35.4865 0.40625 0.174002 0.00174983 2026-02-21T01:54:50.252 INFO:tasks.workunit.client.0.trial009.stdout: 17 16 148334 148318 34.0778 11.5391 0.00138908 0.00183253 2026-02-21T01:54:51.252 INFO:tasks.workunit.client.0.trial009.stdout: 18 16 158058 158042 34.2946 37.9844 0.00161994 0.00182094 2026-02-21T01:54:52.252 INFO:tasks.workunit.client.0.trial009.stdout: 19 16 167382 167366 34.4065 36.4219 0.00160114 0.001815 2026-02-21T01:54:53.252 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:54:53.249225+0000 min lat: 0.000494168 max lat: 0.539214 avg lat: 0.00181001 2026-02-21T01:54:53.252 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T01:54:53.252 INFO:tasks.workunit.client.0.trial009.stdout: 20 16 176679 176663 34.5018 36.3164 0.00167816 0.00181001 2026-02-21T01:54:54.252 INFO:tasks.workunit.client.0.trial009.stdout: 21 16 185998 185982 34.5922 36.4023 0.00133737 0.00180526 2026-02-21T01:54:55.252 INFO:tasks.workunit.client.0.trial009.stdout: 22 16 193147 193131 34.2891 27.9258 0.184808 0.00180366 2026-02-21T01:54:56.252 INFO:tasks.workunit.client.0.trial009.stdout: 23 16 193246 193230 32.815 0.386719 0.193997 0.00189974 2026-02-21T01:54:57.252 INFO:tasks.workunit.client.0.trial009.stdout: 24 16 197323 197307 32.1113 15.9258 0.00122213 0.00194485 2026-02-21T01:54:58.252 INFO:tasks.workunit.client.0.trial009.stdout: 25 15 206686 206671 32.2898 36.5781 0.00204458 0.0019341 2026-02-21T01:54:59.252 INFO:tasks.workunit.client.0.trial009.stdout: 26 16 215958 215942 32.4407 36.2148 0.00191865 0.0019251 2026-02-21T01:55:00.253 INFO:tasks.workunit.client.0.trial009.stdout: 27 16 225136 225120 32.5669 35.8516 0.00336214 0.00191763 2026-02-21T01:55:01.253 INFO:tasks.workunit.client.0.trial009.stdout: 28 16 234156 234140 32.6621 35.2344 0.00155763 0.00191204 2026-02-21T01:55:02.253 INFO:tasks.workunit.client.0.trial009.stdout: 29 16 243514 243498 32.7962 36.5547 0.00195306 0.00190422 2026-02-21T01:55:03.253 INFO:tasks.workunit.client.0.trial009.stdout:Total time run: 30.0009 2026-02-21T01:55:03.253 INFO:tasks.workunit.client.0.trial009.stdout:Total writes made: 252799 2026-02-21T01:55:03.253 INFO:tasks.workunit.client.0.trial009.stdout:Write size: 4096 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Object size: 4096 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Bandwidth (MB/sec): 32.9156 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Bandwidth: 14.0124 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Max bandwidth (MB/sec): 44.0156 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Min bandwidth (MB/sec): 0.28125 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Average IOPS: 8426 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Stddev IOPS: 3587.18 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Max IOPS: 11268 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Min IOPS: 72 2026-02-21T01:55:03.254 INFO:tasks.workunit.client.0.trial009.stdout:Average Latency(s): 0.00189733 2026-02-21T01:55:03.255 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Latency(s): 0.00868339 2026-02-21T01:55:03.255 INFO:tasks.workunit.client.0.trial009.stdout:Max latency(s): 0.539214 2026-02-21T01:55:03.255 INFO:tasks.workunit.client.0.trial009.stdout:Min latency(s): 0.000494168 2026-02-21T01:55:03.257 INFO:tasks.workunit.client.0.trial009.stdout:after bench 2026-02-21T01:55:03.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:57: TEST_bluestore: echo 'after bench' 2026-02-21T01:55:03.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 88624 2026-02-21T01:55:03.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: sleep 1 2026-02-21T01:55:04.258 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 88624 2026-02-21T01:55:04.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 60: kill: (88624) - No such process 2026-02-21T01:55:04.259 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:61: TEST_bluestore: ceph osd down 0 2026-02-21T01:55:04.479 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:55:04.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 91074 2026-02-21T01:55:04.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: sleep 1 2026-02-21T01:55:05.490 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 91074 2026-02-21T01:55:05.491 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 62: kill: (91074) - No such process 2026-02-21T01:55:05.491 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:63: TEST_bluestore: ceph osd down 1 2026-02-21T01:55:05.703 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:55:05.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 93374 2026-02-21T01:55:05.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: sleep 1 2026-02-21T01:55:06.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 93374 2026-02-21T01:55:06.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 64: kill: (93374) - No such process 2026-02-21T01:55:06.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:65: TEST_bluestore: ceph osd down 2 2026-02-21T01:55:06.929 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:55:06.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 95584 2026-02-21T01:55:06.940 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: sleep 1 2026-02-21T01:55:07.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 95584 2026-02-21T01:55:07.941 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 66: kill: (95584) - No such process 2026-02-21T01:55:07.942 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:67: TEST_bluestore: ceph osd down 3 2026-02-21T01:55:08.200 INFO:tasks.workunit.client.0.trial009.stderr:osd.3 is already down. 2026-02-21T01:55:08.210 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:70: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:55:11.602 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:11.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:71: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:55:14.986 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:15.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:72: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:55:18.366 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:18.401 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:73: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:55:21.814 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:21.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:75: TEST_bluestore: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-21T01:55:21.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:76: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-21T01:55:21.871 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:24.672 INFO:tasks.workunit.client.0.trial009.stdout:0 : device size 0x20000000(512 MiB) : using 0x15d00000(349 MiB) 2026-02-21T01:55:24.672 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0xf00000(15 MiB) 2026-02-21T01:55:24.672 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0x100000000(4 GiB) : using 0xae343000(2.7 GiB) 2026-02-21T01:55:24.672 INFO:tasks.workunit.client.0.trial009.stdout:Expanding DB/WAL... 2026-02-21T01:55:24.673 INFO:tasks.workunit.client.0.trial009.stdout:0 : nothing to do, skipped 2026-02-21T01:55:24.674 INFO:tasks.workunit.client.0.trial009.stdout:1 : nothing to do, skipped 2026-02-21T01:55:24.675 INFO:tasks.workunit.client.0.trial009.stdout:2 : Expanding to 0x100000000(4 GiB) 2026-02-21T01:55:25.304 INFO:tasks.workunit.client.0.trial009.stdout:2 : size updated to 0x100000000(4 GiB) 2026-02-21T01:55:28.431 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:77: TEST_bluestore: truncate td/osd-bluefs-volume-ops/1/block -s 11811160064 2026-02-21T01:55:28.432 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:78: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 bluefs-bdev-expand 2026-02-21T01:55:28.452 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:31.210 INFO:tasks.workunit.client.0.trial009.stdout:0 : device size 0x20000000(512 MiB) : using 0x15d00000(349 MiB) 2026-02-21T01:55:31.210 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0xf00000(15 MiB) 2026-02-21T01:55:31.211 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0x2c0000000(11 GiB) : using 0x26e562000(9.7 GiB) 2026-02-21T01:55:31.211 INFO:tasks.workunit.client.0.trial009.stdout:Expanding DB/WAL... 2026-02-21T01:55:31.211 INFO:tasks.workunit.client.0.trial009.stdout:0 : nothing to do, skipped 2026-02-21T01:55:31.211 INFO:tasks.workunit.client.0.trial009.stdout:1 : nothing to do, skipped 2026-02-21T01:55:31.211 INFO:tasks.workunit.client.0.trial009.stdout:2 : Expanding to 0x2c0000000(11 GiB) 2026-02-21T01:55:31.211 INFO:tasks.workunit.client.0.trial009.stdout:2 : size updated to 0x2c0000000(11 GiB) 2026-02-21T01:55:34.301 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:79: TEST_bluestore: truncate td/osd-bluefs-volume-ops/2/block -s 4295099392 2026-02-21T01:55:34.302 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:80: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 bluefs-bdev-expand 2026-02-21T01:55:34.321 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:37.126 INFO:tasks.workunit.client.0.trial009.stdout:0 : device size 0x20000000(512 MiB) : using 0x15d00000(349 MiB) 2026-02-21T01:55:37.126 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0xf00000(15 MiB) 2026-02-21T01:55:37.126 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0x100020000(4.0 GiB) : using 0xae4c4000(2.7 GiB) 2026-02-21T01:55:37.126 INFO:tasks.workunit.client.0.trial009.stdout:Expanding DB/WAL... 2026-02-21T01:55:37.126 INFO:tasks.workunit.client.0.trial009.stdout:0 : nothing to do, skipped 2026-02-21T01:55:37.127 INFO:tasks.workunit.client.0.trial009.stdout:1 : nothing to do, skipped 2026-02-21T01:55:37.127 INFO:tasks.workunit.client.0.trial009.stdout:2 : Expanding to 0x100020000(4.0 GiB) 2026-02-21T01:55:37.127 INFO:tasks.workunit.client.0.trial009.stdout:2 : size updated to 0x100020000(4.0 GiB) 2026-02-21T01:55:40.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:81: TEST_bluestore: truncate td/osd-bluefs-volume-ops/3/block -s 4293918720 2026-02-21T01:55:40.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:82: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 bluefs-bdev-expand 2026-02-21T01:55:40.233 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:0 : device size 0x20000000(512 MiB) : using 0x15d00000(349 MiB) 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0xf00000(15 MiB) 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0xfff00000(4.0 GiB) : using 0xae515000(2.7 GiB) 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:Expanding DB/WAL... 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:0 : nothing to do, skipped 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:1 : nothing to do, skipped 2026-02-21T01:55:42.998 INFO:tasks.workunit.client.0.trial009.stdout:2 : Expanding to 0xfff00000(4.0 GiB) 2026-02-21T01:55:42.999 INFO:tasks.workunit.client.0.trial009.stdout:2 : size updated to 0xfff00000(4.0 GiB) 2026-02-21T01:55:46.170 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:85: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:55:47.122 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:47.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:86: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:55:48.054 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:48.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:87: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:55:48.986 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:49.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:88: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:55:49.926 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:49.963 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:90: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-21T01:55:49.983 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:50.183 INFO:tasks.workunit.client.0.trial009.stdout:0 : device size 0x20000000(512 MiB) : using 0x1900000(25 MiB) 2026-02-21T01:55:50.183 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0x4000000(64 MiB) 2026-02-21T01:55:50.183 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0x100000000(4 GiB) : using 0x2e343000(739 MiB) 2026-02-21T01:55:50.373 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:92: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/block.db --command bluefs-bdev-migrate 2026-02-21T01:55:50.385 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:51.010 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-21T01:55:51.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:97: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:55:51.842 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:51.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:100: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-21T01:55:51.898 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:52.546 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-21T01:55:52.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:105: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:55:53.370 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:53.407 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:108: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block.wal --devs-source td/osd-bluefs-volume-ops/2/block.db --dev-target td/osd-bluefs-volume-ops/2/block --command bluefs-bdev-migrate 2026-02-21T01:55:53.426 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:54.090 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-21T01:55:54.090 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-21T01:55:54.098 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:114: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:55:54.846 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:54.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:117: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block.wal --command bluefs-bdev-migrate 2026-02-21T01:55:54.893 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:55.022 INFO:tasks.workunit.client.0.trial009.stderr:Migrate to WAL device isn't supported. 2026-02-21T01:55:55.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:123: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-21T01:55:55.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:126: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:55:55.910 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:55.948 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:129: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-21T01:55:55.967 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:56.602 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-21T01:55:56.610 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:134: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:55:57.414 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:57.445 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:136: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --dev-target td/osd-bluefs-volume-ops/3/block.db --command bluefs-bdev-migrate 2026-02-21T01:55:57.463 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:55:57.897 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:141: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:55:58.734 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:143: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:55:58.766 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:55:58.767 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:55:58.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T01:55:58.769 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:55:58.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:55:58.787 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:55:58.781+0000 7f62579a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:55:58.789 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:55:58.781+0000 7f62579a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:55:58.790 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:55:58.781+0000 7f62579a4a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:55:58.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:55:58.980 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:55:58.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:55:58.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:55:59.195 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:55:59.239 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:55:59.233+0000 7f62579a4a40 -1 Falling back to public interface 2026-02-21T01:56:00.196 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:00.197 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:00.197 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:56:00.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:56:00.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:00.198 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:56:00.419 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:01.152 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:01.145+0000 7f62579a4a40 -1 osd.0 23 log_to_monitors true 2026-02-21T01:56:01.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:01.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:01.420 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:56:01.421 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:56:01.421 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:01.421 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:56:01.634 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 31 up_thru 31 down_at 23 last_clean_interval [5,23) [v2:127.0.0.1:6802/3509696015,v1:127.0.0.1:6803/3509696015] [v2:127.0.0.1:6804/3509696015,v1:127.0.0.1:6805/3509696015] exists,up 52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T01:56:01.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:56:01.635 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:56:01.635 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:56:01.635 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T01:56:01.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: osd_pid0=107195 2026-02-21T01:56:01.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:145: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-21T01:56:01.638 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:56:01.639 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:56:01.641 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:56:01.642 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-21T01:56:01.642 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:56:01.643 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:56:01.663 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:01.657+0000 7f6dcd6e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:01.664 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:01.657+0000 7f6dcd6e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:01.665 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:01.657+0000 7f6dcd6e7a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:01.855 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:56:01.855 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:56:01.856 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:56:01.856 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:56:01.856 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:56:01.856 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:01.856 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:56:01.857 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:56:01.857 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:01.857 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:56:02.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:02.119 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:02.113+0000 7f6dcd6e7a40 -1 Falling back to public interface 2026-02-21T01:56:03.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:03.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:03.067 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:56:03.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:56:03.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:03.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:56:03.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:03.887 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:03.881+0000 7f6dcd6e7a40 -1 osd.1 24 log_to_monitors true 2026-02-21T01:56:04.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:04.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:04.288 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:56:04.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:56:04.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:04.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:56:04.494 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 34 up_thru 34 down_at 25 last_clean_interval [10,24) [v2:127.0.0.1:6810/4050919410,v1:127.0.0.1:6811/4050919410] [v2:127.0.0.1:6812/4050919410,v1:127.0.0.1:6813/4050919410] exists,up 78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T01:56:04.494 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:56:04.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:56:04.495 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:56:04.495 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: osd_pid1=108136 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:147: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:56:04.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:56:04.499 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:56:04.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:56:04.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:56:04.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-21T01:56:04.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:56:04.501 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:56:04.502 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:56:04.502 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-21T01:56:04.503 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:56:04.503 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:56:04.503 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:56:04.503 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:56:04.518 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:04.513+0000 7f683b45fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:04.519 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:04.513+0000 7f683b45fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:04.520 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:04.513+0000 7f683b45fa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:56:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:56:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:56:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:56:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:56:04.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:04.717 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:56:04.717 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:56:04.717 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:04.718 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:56:04.924 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:04.955 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:04.949+0000 7f683b45fa40 -1 Falling back to public interface 2026-02-21T01:56:05.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:05.926 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:05.927 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:56:05.927 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:56:05.927 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:05.927 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:56:06.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:06.654 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:06.645+0000 7f683b45fa40 -1 osd.2 26 log_to_monitors true 2026-02-21T01:56:07.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:07.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:07.134 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:56:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:56:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:07.134 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:56:07.349 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 37 up_thru 37 down_at 27 last_clean_interval [15,26) [v2:127.0.0.1:6818/1916120935,v1:127.0.0.1:6819/1916120935] [v2:127.0.0.1:6820/1916120935,v1:127.0.0.1:6821/1916120935] exists,up 130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T01:56:07.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:56:07.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:56:07.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:56:07.350 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: osd_pid2=109077 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:149: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:56:07.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:56:07.354 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:56:07.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:56:07.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-21T01:56:07.357 INFO:tasks.workunit.client.0.trial009.stdout:start osd.3 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:56:07.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:56:07.372 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:07.365+0000 7f0bd6b48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:07.374 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:07.369+0000 7f0bd6b48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:07.375 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:07.369+0000 7f0bd6b48a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:07.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:56:07.578 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:56:07.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:07.578 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:56:07.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:07.891 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:07.885+0000 7f0bd6b48a40 -1 Falling back to public interface 2026-02-21T01:56:08.787 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:08.788 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:08.788 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:56:08.789 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:56:08.789 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:08.789 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:56:09.004 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:56:09.709 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:56:09.701+0000 7f0bd6b48a40 -1 osd.3 28 log_to_monitors true 2026-02-21T01:56:10.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:56:10.005 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:56:10.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:56:10.006 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:56:10.006 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:56:10.007 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:56:10.217 INFO:tasks.workunit.client.0.trial009.stdout:osd.3 up in weight 1 up_from 42 up_thru 42 down_at 29 last_clean_interval [19,28) [v2:127.0.0.1:6826/2307927946,v1:127.0.0.1:6827/2307927946] [v2:127.0.0.1:6828/2307927946,v1:127.0.0.1:6829/2307927946] exists,up e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T01:56:10.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:56:10.217 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:56:10.218 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:56:10.218 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-21T01:56:10.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: osd_pid3=110186 2026-02-21T01:56:10.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:152: TEST_bluestore: wait_for_clean 2026-02-21T01:56:10.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T01:56:10.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T01:56:10.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T01:56:10.220 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T01:56:10.220 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T01:56:10.220 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T01:56:10.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T01:56:10.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T01:56:10.221 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T01:56:10.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T01:56:10.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T01:56:10.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T01:56:10.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T01:56:10.318 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T01:56:10.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T01:56:10.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T01:56:10.530 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T01:56:10.530 INFO:tasks.workunit.client.0.trial009.stderr:2 2026-02-21T01:56:10.530 INFO:tasks.workunit.client.0.trial009.stderr:3' 2026-02-21T01:56:10.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T01:56:10.531 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:56:10.531 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T01:56:10.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=133143986180 2026-02-21T01:56:10.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 133143986180 2026-02-21T01:56:10.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180' 2026-02-21T01:56:10.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:56:10.611 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T01:56:10.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-21T01:56:10.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-21T01:56:10.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180 1-146028888068' 2026-02-21T01:56:10.691 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:56:10.692 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T01:56:10.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789955 2026-02-21T01:56:10.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789955 2026-02-21T01:56:10.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180 1-146028888068 2-158913789955' 2026-02-21T01:56:10.770 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T01:56:10.770 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-21T01:56:10.846 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-21T01:56:10.846 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-21T01:56:10.846 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-133143986180 1-146028888068 2-158913789955 3-180388626435' 2026-02-21T01:56:10.846 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:56:10.847 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-133143986180 2026-02-21T01:56:10.847 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:56:10.848 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T01:56:10.849 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-133143986180 2026-02-21T01:56:10.849 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:56:10.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=133143986180 2026-02-21T01:56:10.850 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 133143986180 2026-02-21T01:56:10.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 133143986180' 2026-02-21T01:56:10.851 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:56:11.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986178 -lt 133143986180 2026-02-21T01:56:11.067 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T01:56:12.068 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T01:56:12.068 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T01:56:12.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 133143986181 -lt 133143986180 2026-02-21T01:56:12.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:56:12.280 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-146028888068 2026-02-21T01:56:12.281 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:56:12.282 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T01:56:12.283 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-146028888068 2026-02-21T01:56:12.283 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:56:12.284 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888068 2026-02-21T01:56:12.285 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 146028888068 2026-02-21T01:56:12.285 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 146028888068' 2026-02-21T01:56:12.285 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T01:56:12.507 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-21T01:56:12.507 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:56:12.508 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-158913789955 2026-02-21T01:56:12.508 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:56:12.509 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T01:56:12.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-158913789955 2026-02-21T01:56:12.510 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:56:12.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789955 2026-02-21T01:56:12.512 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 158913789955' 2026-02-21T01:56:12.512 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 158913789955 2026-02-21T01:56:12.512 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T01:56:12.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-21T01:56:12.731 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T01:56:12.731 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-180388626435 2026-02-21T01:56:12.732 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T01:56:12.733 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-21T01:56:12.733 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-180388626435 2026-02-21T01:56:12.734 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T01:56:12.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626435 2026-02-21T01:56:12.735 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 180388626435' 2026-02-21T01:56:12.735 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.3 seq 180388626435 2026-02-21T01:56:12.736 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T01:56:12.959 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-21T01:56:12.960 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T01:56:12.961 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:56:12.961 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:56:13.240 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T01:56:13.241 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T01:56:13.241 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T01:56:13.242 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T01:56:13.242 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T01:56:13.242 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T01:56:13.242 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T01:56:13.242 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T01:56:13.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T01:56:13.455 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T01:56:13.456 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T01:56:13.456 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T01:56:13.742 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T01:56:13.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T01:56:13.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T01:56:13.743 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:155: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-21T01:56:13.761 INFO:tasks.workunit.client.0.trial009.stdout:hints = 1 2026-02-21T01:56:13.761 INFO:tasks.workunit.client.0.trial009.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-21T01:56:13.762 INFO:tasks.workunit.client.0.trial009.stdout:Object prefix: benchmark_data_trial009_111495 2026-02-21T01:56:13.762 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T01:56:13.762 INFO:tasks.workunit.client.0.trial009.stdout: 0 0 0 0 0 0 - 0 2026-02-21T01:56:14.761 INFO:tasks.workunit.client.0.trial009.stdout: 1 16 10906 10890 42.536 42.5391 0.000913297 0.00146661 2026-02-21T01:56:15.761 INFO:tasks.workunit.client.0.trial009.stdout: 2 16 21760 21744 42.4656 42.3984 0.00148237 0.0014697 2026-02-21T01:56:16.761 INFO:tasks.workunit.client.0.trial009.stdout: 3 16 32073 32057 41.7377 40.2852 0.00114188 0.00149557 2026-02-21T01:56:17.762 INFO:tasks.workunit.client.0.trial009.stdout: 4 15 42228 42213 41.2204 39.6719 0.00168539 0.00151441 2026-02-21T01:56:18.762 INFO:tasks.workunit.client.0.trial009.stdout: 5 16 52483 52467 40.9867 40.0547 0.00151231 0.00152314 2026-02-21T01:56:19.762 INFO:tasks.workunit.client.0.trial009.stdout: 6 16 62413 62397 40.6199 38.7891 0.00128955 0.00153699 2026-02-21T01:56:20.762 INFO:tasks.workunit.client.0.trial009.stdout: 7 16 72456 72440 40.421 39.2305 0.00137832 0.0015445 2026-02-21T01:56:21.762 INFO:tasks.workunit.client.0.trial009.stdout: 8 16 82582 82566 40.3119 39.5547 0.00290274 0.00154873 2026-02-21T01:56:22.762 INFO:tasks.workunit.client.0.trial009.stdout: 9 15 92694 92679 40.2218 39.5039 0.0017291 0.00155232 2026-02-21T01:56:23.762 INFO:tasks.workunit.client.0.trial009.stdout: 10 16 102739 102723 40.1227 39.2344 0.00175118 0.00155611 2026-02-21T01:56:24.762 INFO:tasks.workunit.client.0.trial009.stdout: 11 15 112669 112654 40.0016 38.793 0.00106847 0.00156087 2026-02-21T01:56:25.762 INFO:tasks.workunit.client.0.trial009.stdout: 12 16 122230 122214 39.7799 37.3438 0.0014457 0.00156951 2026-02-21T01:56:26.762 INFO:tasks.workunit.client.0.trial009.stdout: 13 16 131870 131854 39.6163 37.6562 0.0013762 0.00157606 2026-02-21T01:56:27.762 INFO:tasks.workunit.client.0.trial009.stdout: 14 16 141508 141492 39.4755 37.6484 0.00162971 0.00158165 2026-02-21T01:56:28.763 INFO:tasks.workunit.client.0.trial009.stdout: 15 16 151039 151023 39.3257 37.2305 0.00236124 0.00158776 2026-02-21T01:56:29.763 INFO:tasks.workunit.client.0.trial009.stdout: 16 16 160124 160108 39.0857 35.4883 0.00504405 0.00159404 2026-02-21T01:56:30.763 INFO:tasks.workunit.client.0.trial009.stdout: 17 16 160726 160710 36.9248 2.35156 0.00166975 0.00169107 2026-02-21T01:56:31.763 INFO:tasks.workunit.client.0.trial009.stdout: 18 16 170234 170218 36.9367 37.1406 0.00183212 0.00169055 2026-02-21T01:56:32.763 INFO:tasks.workunit.client.0.trial009.stdout: 19 16 179837 179821 36.9668 37.5117 0.00179607 0.00168916 2026-02-21T01:56:33.763 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:56:33.760095+0000 min lat: 0.000558087 max lat: 0.158792 avg lat: 0.00168826 2026-02-21T01:56:33.763 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T01:56:33.763 INFO:tasks.workunit.client.0.trial009.stdout: 20 16 189403 189387 36.9867 37.3672 0.00146979 0.00168826 2026-02-21T01:56:34.763 INFO:tasks.workunit.client.0.trial009.stdout: 21 16 198987 198971 37.008 37.4375 0.00140037 0.00168728 2026-02-21T01:56:35.763 INFO:tasks.workunit.client.0.trial009.stdout: 22 16 208627 208611 37.0373 37.6562 0.00113995 0.00168597 2026-02-21T01:56:36.763 INFO:tasks.workunit.client.0.trial009.stdout: 23 16 209914 209898 35.6456 5.02734 0.052314 0.00174404 2026-02-21T01:56:37.763 INFO:tasks.workunit.client.0.trial009.stdout: 24 16 210115 210099 34.1931 0.785156 0.179311 0.00182416 2026-02-21T01:56:38.763 INFO:tasks.workunit.client.0.trial009.stdout: 25 16 210427 210411 32.8741 1.21875 0.0573799 0.00189678 2026-02-21T01:56:39.763 INFO:tasks.workunit.client.0.trial009.stdout: 26 16 210555 210539 31.6289 0.5 0.186421 0.00196953 2026-02-21T01:56:40.763 INFO:tasks.workunit.client.0.trial009.stdout: 27 16 210644 210628 30.4704 0.347656 0.193314 0.00204425 2026-02-21T01:56:41.764 INFO:tasks.workunit.client.0.trial009.stdout: 28 16 210743 210727 29.396 0.386719 0.0856326 0.00211885 2026-02-21T01:56:42.764 INFO:tasks.workunit.client.0.trial009.stdout: 29 16 217329 217313 29.2694 25.7266 0.00149598 0.00213381 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Total time run: 30.0012 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Total writes made: 226831 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Write size: 4096 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Object size: 4096 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Bandwidth (MB/sec): 29.5341 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Bandwidth: 16.1979 2026-02-21T01:56:43.764 INFO:tasks.workunit.client.0.trial009.stdout:Max bandwidth (MB/sec): 42.5391 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Min bandwidth (MB/sec): 0.347656 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Average IOPS: 7560 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Stddev IOPS: 4146.65 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Max IOPS: 10890 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Min IOPS: 89 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Average Latency(s): 0.00211471 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Latency(s): 0.0077437 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Max latency(s): 0.255975 2026-02-21T01:56:43.765 INFO:tasks.workunit.client.0.trial009.stdout:Min latency(s): 0.000558087 2026-02-21T01:56:43.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 107195 2026-02-21T01:56:43.768 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: sleep 1 2026-02-21T01:56:44.769 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 107195 2026-02-21T01:56:44.769 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 158: kill: (107195) - No such process 2026-02-21T01:56:44.769 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:159: TEST_bluestore: ceph osd down 0 2026-02-21T01:56:44.984 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:56:44.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 108136 2026-02-21T01:56:44.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: sleep 1 2026-02-21T01:56:45.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 108136 2026-02-21T01:56:45.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 160: kill: (108136) - No such process 2026-02-21T01:56:45.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:161: TEST_bluestore: ceph osd down 1 2026-02-21T01:56:46.204 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:56:46.213 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 109077 2026-02-21T01:56:46.214 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: sleep 1 2026-02-21T01:56:47.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 109077 2026-02-21T01:56:47.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 162: kill: (109077) - No such process 2026-02-21T01:56:47.215 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:163: TEST_bluestore: ceph osd down 2 2026-02-21T01:56:47.415 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:56:47.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 110186 2026-02-21T01:56:47.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: sleep 1 2026-02-21T01:56:48.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 110186 2026-02-21T01:56:48.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 164: kill: (110186) - No such process 2026-02-21T01:56:48.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:165: TEST_bluestore: ceph osd down 3 2026-02-21T01:56:48.663 INFO:tasks.workunit.client.0.trial009.stderr:osd.3 is already down. 2026-02-21T01:56:48.672 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:168: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:56:51.646 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:56:51.687 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:170: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal count=512 bs=1M 2026-02-21T01:56:52.010 INFO:tasks.workunit.client.0.trial009.stderr:512+0 records in 2026-02-21T01:56:52.010 INFO:tasks.workunit.client.0.trial009.stderr:512+0 records out 2026-02-21T01:56:52.010 INFO:tasks.workunit.client.0.trial009.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.321377 s, 1.7 GB/s 2026-02-21T01:56:52.010 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:171: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --dev-target td/osd-bluefs-volume-ops/0/wal --command bluefs-bdev-new-wal 2026-02-21T01:56:52.028 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:56:54.734 INFO:tasks.workunit.client.0.trial009.stdout:WAL device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal 2026-02-21T01:56:54.758 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:175: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:56:57.822 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:56:57.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:178: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:57:01.190 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:57:01.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:180: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db count=1024 bs=1M 2026-02-21T01:57:01.849 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records in 2026-02-21T01:57:01.849 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records out 2026-02-21T01:57:01.850 INFO:tasks.workunit.client.0.trial009.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.616557 s, 1.7 GB/s 2026-02-21T01:57:01.850 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:181: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --dev-target td/osd-bluefs-volume-ops/1/db --command bluefs-bdev-new-db 2026-02-21T01:57:01.868 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:05.230 INFO:tasks.workunit.client.0.trial009.stdout:DB device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db 2026-02-21T01:57:05.253 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:185: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block --dev-target td/osd-bluefs-volume-ops/1/block.db --command bluefs-bdev-migrate 2026-02-21T01:57:05.270 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:08.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:190: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:57:11.406 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:57:11.453 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:193: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:57:14.942 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:57:14.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:195: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-db 2026-02-21T01:57:14.997 INFO:tasks.workunit.client.0.trial009.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-21T01:57:14.997 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:17.870 INFO:tasks.workunit.client.0.trial009.stdout:DB device added 2026-02-21T01:57:17.886 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:198: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-wal 2026-02-21T01:57:17.901 INFO:tasks.workunit.client.0.trial009.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-21T01:57:17.901 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:20.910 INFO:tasks.workunit.client.0.trial009.stdout:WAL device added 2026-02-21T01:57:20.931 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:201: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/block.db --command bluefs-bdev-migrate 2026-02-21T01:57:20.947 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:24.476 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:206: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:57:28.154 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:57:28.201 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:209: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:57:31.162 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:57:31.197 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:211: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --command bluefs-bdev-new-wal 2026-02-21T01:57:31.216 INFO:tasks.workunit.client.0.trial009.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-21T01:57:31.216 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:33.530 INFO:tasks.workunit.client.0.trial009.stdout:WAL device added 2026-02-21T01:57:33.551 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:214: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-21T01:57:33.567 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:57:36.474 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block.db 2026-02-21T01:57:36.502 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:219: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:57:39.414 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:57:39.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:221: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-21T01:57:39.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:57:39.454 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:57:39.455 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:57:39.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T01:57:39.457 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:57:39.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:57:39.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:57:39.459 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-21T01:57:39.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:57:39.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:57:39.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:57:39.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:57:39.477 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:39.467+0000 7f5d3b4b8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:39.479 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:39.471+0000 7f5d3b4b8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:39.480 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:39.471+0000 7f5d3b4b8a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:39.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:57:39.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:57:39.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:57:39.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:57:39.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:57:39.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:39.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:57:39.666 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:57:39.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:39.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:39.870 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:40.107 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:40.099+0000 7f5d3b4b8a40 -1 Falling back to public interface 2026-02-21T01:57:40.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:40.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:40.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:57:40.872 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:57:40.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:40.873 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:41.088 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:42.090 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:42.090 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:42.090 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:57:42.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:57:42.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:42.091 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:42.303 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:43.305 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:43.305 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:43.305 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T01:57:43.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T01:57:43.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:43.306 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:43.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:44.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:44.524 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:44.525 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T01:57:44.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T01:57:44.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:44.525 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:44.736 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:45.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:45.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:45.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T01:57:45.738 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T01:57:45.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:45.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:45.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:46.336 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:46.327+0000 7f5d3b4b8a40 -1 osd.0 45 log_to_monitors true 2026-02-21T01:57:46.956 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:46.956 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:46.957 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T01:57:46.957 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T01:57:46.958 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:46.958 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:57:47.160 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 53 up_thru 53 down_at 45 last_clean_interval [31,45) [v2:127.0.0.1:6802/599214240,v1:127.0.0.1:6803/599214240] [v2:127.0.0.1:6804/599214240,v1:127.0.0.1:6805/599214240] exists,up 52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T01:57:47.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:57:47.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:57:47.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:57:47.161 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T01:57:47.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: osd_pid0=117645 2026-02-21T01:57:47.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:223: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-21T01:57:47.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:57:47.163 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:57:47.164 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:57:47.165 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:57:47.166 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:57:47.166 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:57:47.167 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-21T01:57:47.168 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:57:47.169 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:57:47.184 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:47.175+0000 7fd4292baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:47.185 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:47.175+0000 7fd4292baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:47.186 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:47.179+0000 7fd4292baa40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:47.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T01:57:47.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:57:47.382 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T01:57:47.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:57:47.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:57:47.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:47.383 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:57:47.383 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:57:47.384 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:47.384 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:47.599 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:47.728 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:47.719+0000 7fd4292baa40 -1 Falling back to public interface 2026-02-21T01:57:48.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:48.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:48.601 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:57:48.602 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:57:48.602 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:48.602 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:48.810 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:49.812 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:49.812 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:49.812 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:57:49.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:57:49.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:49.813 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:50.015 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:51.016 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:51.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:51.017 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T01:57:51.017 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T01:57:51.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:51.018 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:51.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:52.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:52.220 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:52.221 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T01:57:52.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T01:57:52.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:52.222 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:52.423 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:53.424 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:53.425 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:53.425 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T01:57:53.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T01:57:53.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:53.426 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:53.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:54.635 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:54.636 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:54.636 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T01:57:54.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T01:57:54.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:54.637 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:54.706 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:54.695+0000 7fd4292baa40 -1 osd.1 47 log_to_monitors true 2026-02-21T01:57:54.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:55.853 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:55.853 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:55.854 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T01:57:55.854 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T01:57:55.854 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:55.855 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T01:57:56.063 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 56 up_thru 56 down_at 47 last_clean_interval [34,47) [v2:127.0.0.1:6810/3836172871,v1:127.0.0.1:6811/3836172871] [v2:127.0.0.1:6812/3836172871,v1:127.0.0.1:6813/3836172871] exists,up 78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T01:57:56.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:57:56.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:57:56.065 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:57:56.065 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-21T01:57:56.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: osd_pid1=118691 2026-02-21T01:57:56.071 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:225: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-21T01:57:56.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:57:56.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:57:56.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T01:57:56.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:57:56.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-21T01:57:56.072 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:57:56.073 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:57:56.074 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:57:56.075 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:57:56.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:57:56.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:57:56.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-21T01:57:56.076 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T01:57:56.076 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T01:57:56.077 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:57:56.077 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-21T01:57:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T01:57:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:57:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:57:56.078 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:57:56.091 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:56.083+0000 7f78d6050a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:56.092 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:56.083+0000 7f78d6050a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:56.093 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:56.083+0000 7f78d6050a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:57:56.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T01:57:56.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:57:56.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T01:57:56.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:57:56.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:57:56.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:56.277 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:57:56.278 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:57:56.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:56.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:57:56.480 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:56.587 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:57:56.579+0000 7f78d6050a40 -1 Falling back to public interface 2026-02-21T01:57:57.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:57.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:57.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:57:57.482 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:57:57.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:57.482 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:57:57.694 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:58.695 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:58.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:58.696 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:57:58.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:57:58.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:58.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:57:58.906 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:57:59.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:57:59.908 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:57:59.908 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T01:57:59.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T01:57:59.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:57:59.909 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:58:00.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:01.124 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:01.124 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T01:58:01.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:01.125 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T01:58:01.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:01.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:58:01.327 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:02.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:02.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:02.329 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T01:58:02.329 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T01:58:02.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:02.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:58:02.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:03.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:03.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:03.531 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T01:58:03.531 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T01:58:03.532 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:03.532 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:58:03.736 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:04.129 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:58:04.123+0000 7f78d6050a40 -1 osd.2 48 log_to_monitors true 2026-02-21T01:58:04.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:04.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:04.738 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T01:58:04.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T01:58:04.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:04.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T01:58:04.945 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 59 up_thru 59 down_at 49 last_clean_interval [37,48) [v2:127.0.0.1:6818/3443090401,v1:127.0.0.1:6819/3443090401] [v2:127.0.0.1:6820/3443090401,v1:127.0.0.1:6821/3443090401] exists,up 130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T01:58:04.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:58:04.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:58:04.946 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:58:04.946 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: osd_pid2=119762 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:227: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-21T01:58:04.949 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:58:04.950 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:58:04.951 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:58:04.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:58:04.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:58:04.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:58:04.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:58:04.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T01:58:04.953 INFO:tasks.workunit.client.0.trial009.stdout:start osd.3 2026-02-21T01:58:04.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:58:04.954 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-21T01:58:04.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T01:58:04.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:58:04.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:58:04.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:58:04.973 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:58:04.967+0000 7fc4855c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:58:04.974 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:58:04.971+0000 7fc4855c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:58:04.975 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:58:04.971+0000 7fc4855c0a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:58:05.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T01:58:05.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:58:05.160 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T01:58:05.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:58:05.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:58:05.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:05.161 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:58:05.161 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:58:05.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:05.162 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:05.374 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:05.464 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:58:05.459+0000 7fc4855c0a40 -1 Falling back to public interface 2026-02-21T01:58:06.375 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:06.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:06.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:58:06.376 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:58:06.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:06.377 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:06.580 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:07.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:07.582 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:07.582 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:58:07.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:58:07.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:07.583 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:07.783 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:08.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:08.785 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:08.785 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T01:58:08.785 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T01:58:08.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:08.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:08.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:09.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:09.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:09.987 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T01:58:09.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T01:58:09.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:09.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:10.190 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:11.192 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:11.192 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:11.192 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T01:58:11.192 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T01:58:11.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:11.193 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:11.394 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:58:11.852 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:58:11.846+0000 7fc4855c0a40 -1 osd.3 50 log_to_monitors true 2026-02-21T01:58:12.395 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:58:12.395 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:58:12.396 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T01:58:12.396 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T01:58:12.396 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:58:12.396 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T01:58:12.607 INFO:tasks.workunit.client.0.trial009.stdout:osd.3 up in weight 1 up_from 62 up_thru 62 down_at 51 last_clean_interval [42,50) [v2:127.0.0.1:6826/734441449,v1:127.0.0.1:6827/734441449] [v2:127.0.0.1:6828/734441449,v1:127.0.0.1:6829/734441449] exists,up e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T01:58:12.608 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T01:58:12.608 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T01:58:12.608 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T01:58:12.608 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-21T01:58:12.609 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: osd_pid3=120833 2026-02-21T01:58:12.609 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:231: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-21T01:58:12.627 INFO:tasks.workunit.client.0.trial009.stdout:hints = 1 2026-02-21T01:58:12.627 INFO:tasks.workunit.client.0.trial009.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-21T01:58:12.627 INFO:tasks.workunit.client.0.trial009.stdout:Object prefix: benchmark_data_trial009_121859 2026-02-21T01:58:12.627 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T01:58:12.627 INFO:tasks.workunit.client.0.trial009.stdout: 0 0 0 0 0 0 - 0 2026-02-21T01:58:13.627 INFO:tasks.workunit.client.0.trial009.stdout: 1 15 11031 11016 43.0273 43.0312 0.00121515 0.00144997 2026-02-21T01:58:14.627 INFO:tasks.workunit.client.0.trial009.stdout: 2 16 21536 21520 42.0276 41.0312 0.0012623 0.00148486 2026-02-21T01:58:15.627 INFO:tasks.workunit.client.0.trial009.stdout: 3 16 32030 32014 41.6814 40.9922 0.00118053 0.00149761 2026-02-21T01:58:16.627 INFO:tasks.workunit.client.0.trial009.stdout: 4 16 42704 42688 41.6834 41.6953 0.00146868 0.00149757 2026-02-21T01:58:17.627 INFO:tasks.workunit.client.0.trial009.stdout: 5 16 53405 53389 41.7062 41.8008 0.00150398 0.00149682 2026-02-21T01:58:18.627 INFO:tasks.workunit.client.0.trial009.stdout: 6 16 63700 63684 41.4572 40.2148 0.00117662 0.00150585 2026-02-21T01:58:19.628 INFO:tasks.workunit.client.0.trial009.stdout: 7 16 74262 74246 41.4283 41.2578 0.000983959 0.00150685 2026-02-21T01:58:20.628 INFO:tasks.workunit.client.0.trial009.stdout: 8 16 76228 76212 37.2097 7.67969 0.159643 0.00166367 2026-02-21T01:58:21.628 INFO:tasks.workunit.client.0.trial009.stdout: 9 16 84986 84970 36.8762 34.2109 0.00145181 0.00169323 2026-02-21T01:58:22.628 INFO:tasks.workunit.client.0.trial009.stdout: 10 16 95532 95516 37.3078 41.1953 0.00154308 0.00167357 2026-02-21T01:58:23.628 INFO:tasks.workunit.client.0.trial009.stdout: 11 16 105929 105913 37.608 40.6133 0.00125857 0.00166024 2026-02-21T01:58:24.628 INFO:tasks.workunit.client.0.trial009.stdout: 12 16 115714 115698 37.659 38.2227 0.00190808 0.00165805 2026-02-21T01:58:25.628 INFO:tasks.workunit.client.0.trial009.stdout: 13 16 125483 125467 37.6973 38.1602 0.00178092 0.00165634 2026-02-21T01:58:26.628 INFO:tasks.workunit.client.0.trial009.stdout: 14 16 135298 135282 37.743 38.3398 0.00130892 0.00165433 2026-02-21T01:58:27.628 INFO:tasks.workunit.client.0.trial009.stdout: 15 15 145363 145348 37.8479 39.3203 0.00204252 0.00164977 2026-02-21T01:58:28.628 INFO:tasks.workunit.client.0.trial009.stdout: 16 15 155407 155392 37.9344 39.2344 0.00127869 0.001646 2026-02-21T01:58:29.628 INFO:tasks.workunit.client.0.trial009.stdout: 17 16 165207 165191 37.9544 38.2773 0.00115083 0.00164515 2026-02-21T01:58:30.628 INFO:tasks.workunit.client.0.trial009.stdout: 18 16 175039 175023 37.9793 38.4062 0.00163849 0.00164405 2026-02-21T01:58:31.628 INFO:tasks.workunit.client.0.trial009.stdout: 19 16 179503 179487 36.8981 17.4375 0.0455474 0.00169036 2026-02-21T01:58:32.628 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T01:58:32.625645+0000 min lat: 0.000495221 max lat: 0.25195 avg lat: 0.00176285 2026-02-21T01:58:32.629 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T01:58:32.629 INFO:tasks.workunit.client.0.trial009.stdout: 20 16 181394 181378 35.4225 7.38672 0.00184944 0.00176285 2026-02-21T01:58:33.629 INFO:tasks.workunit.client.0.trial009.stdout: 21 16 191437 191421 35.6037 39.2305 0.00108742 0.00175387 2026-02-21T01:58:34.629 INFO:tasks.workunit.client.0.trial009.stdout: 22 16 201556 201540 35.7819 39.5273 0.00175769 0.00174513 2026-02-21T01:58:35.629 INFO:tasks.workunit.client.0.trial009.stdout: 23 16 211483 211467 35.912 38.7773 0.00216072 0.00173879 2026-02-21T01:58:36.629 INFO:tasks.workunit.client.0.trial009.stdout: 24 16 221260 221244 36.0068 38.1914 0.000934668 0.00173422 2026-02-21T01:58:37.629 INFO:tasks.workunit.client.0.trial009.stdout: 25 16 229620 229604 35.8727 32.6562 0.135516 0.00173187 2026-02-21T01:58:38.629 INFO:tasks.workunit.client.0.trial009.stdout: 26 16 229641 229625 34.4962 0.0820312 0.827621 0.00177009 2026-02-21T01:58:39.629 INFO:tasks.workunit.client.0.trial009.stdout: 27 16 229754 229738 33.2349 0.441406 0.138528 0.00187534 2026-02-21T01:58:40.629 INFO:tasks.workunit.client.0.trial009.stdout: 28 16 236885 236869 33.0427 27.8555 0.00162832 0.00188996 2026-02-21T01:58:41.629 INFO:tasks.workunit.client.0.trial009.stdout: 29 16 246752 246736 33.2323 38.543 0.00112951 0.00187917 2026-02-21T01:58:42.629 INFO:tasks.workunit.client.0.trial009.stdout:Total time run: 30.0009 2026-02-21T01:58:42.629 INFO:tasks.workunit.client.0.trial009.stdout:Total writes made: 256484 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Write size: 4096 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Object size: 4096 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Bandwidth (MB/sec): 33.3953 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Bandwidth: 12.9941 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Max bandwidth (MB/sec): 43.0312 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Min bandwidth (MB/sec): 0.0820312 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Average IOPS: 8549 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Stddev IOPS: 3326.48 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Max IOPS: 11016 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Min IOPS: 21 2026-02-21T01:58:42.630 INFO:tasks.workunit.client.0.trial009.stdout:Average Latency(s): 0.00187 2026-02-21T01:58:42.631 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Latency(s): 0.0089817 2026-02-21T01:58:42.631 INFO:tasks.workunit.client.0.trial009.stdout:Max latency(s): 0.992941 2026-02-21T01:58:42.631 INFO:tasks.workunit.client.0.trial009.stdout:Min latency(s): 0.000495221 2026-02-21T01:58:42.632 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 117645 2026-02-21T01:58:42.633 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: sleep 1 2026-02-21T01:58:43.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 117645 2026-02-21T01:58:43.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 234: kill: (117645) - No such process 2026-02-21T01:58:43.634 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:235: TEST_bluestore: ceph osd down 0 2026-02-21T01:58:43.840 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T01:58:43.849 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 118691 2026-02-21T01:58:43.849 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-21T01:58:44.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 118691 2026-02-21T01:58:44.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 236: kill: (118691) - No such process 2026-02-21T01:58:44.851 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:237: TEST_bluestore: ceph osd down 1 2026-02-21T01:58:45.579 INFO:tasks.workunit.client.0.trial009.stderr:osd.1 is already down. 2026-02-21T01:58:45.592 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 119762 2026-02-21T01:58:45.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: sleep 1 2026-02-21T01:58:46.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 119762 2026-02-21T01:58:46.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 238: kill: (119762) - No such process 2026-02-21T01:58:46.594 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:239: TEST_bluestore: ceph osd down 2 2026-02-21T01:58:46.799 INFO:tasks.workunit.client.0.trial009.stderr:osd.2 is already down. 2026-02-21T01:58:46.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 120833 2026-02-21T01:58:46.809 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: sleep 1 2026-02-21T01:58:47.810 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 120833 2026-02-21T01:58:47.810 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 240: kill: (120833) - No such process 2026-02-21T01:58:47.811 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:241: TEST_bluestore: ceph osd down 3 2026-02-21T01:58:48.071 INFO:tasks.workunit.client.0.trial009.stderr:osd.3 is already down. 2026-02-21T01:58:48.083 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:244: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:58:52.306 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:58:52.369 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:246: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/db2 count=1024 bs=1M 2026-02-21T01:58:53.128 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records in 2026-02-21T01:58:53.129 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records out 2026-02-21T01:58:53.129 INFO:tasks.workunit.client.0.trial009.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.758092 s, 1.4 GB/s 2026-02-21T01:58:53.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:247: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/db2 --command bluefs-bdev-migrate 2026-02-21T01:58:53.147 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:58:57.262 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-21T01:58:57.262 INFO:tasks.workunit.client.0.trial009.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/db2 2026-02-21T01:58:57.286 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:252: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:59:01.434 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:01.489 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:256: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal2 count=512 bs=1M 2026-02-21T01:59:01.800 INFO:tasks.workunit.client.0.trial009.stderr:512+0 records in 2026-02-21T01:59:01.800 INFO:tasks.workunit.client.0.trial009.stderr:512+0 records out 2026-02-21T01:59:01.800 INFO:tasks.workunit.client.0.trial009.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.309744 s, 1.7 GB/s 2026-02-21T01:59:01.800 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:257: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/wal2 --command bluefs-bdev-migrate 2026-02-21T01:59:01.818 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:59:05.750 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-21T01:59:05.750 INFO:tasks.workunit.client.0.trial009.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal2 2026-02-21T01:59:05.781 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:261: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/0/wal 2026-02-21T01:59:05.849 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:263: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T01:59:10.002 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:10.064 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:266: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:59:14.142 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:14.204 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:268: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db2 count=1024 bs=1M 2026-02-21T01:59:14.823 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records in 2026-02-21T01:59:14.824 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records out 2026-02-21T01:59:14.824 INFO:tasks.workunit.client.0.trial009.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.618366 s, 1.7 GB/s 2026-02-21T01:59:14.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:269: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --devs-source td/osd-bluefs-volume-ops/1/block.wal --dev-target td/osd-bluefs-volume-ops/1/db2 --command bluefs-bdev-migrate 2026-02-21T01:59:14.841 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:59:19.282 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-21T01:59:19.282 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/1/block.wal 2026-02-21T01:59:19.282 INFO:tasks.workunit.client.0.trial009.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db2 2026-02-21T01:59:19.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:275: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db 2026-02-21T01:59:19.442 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:277: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:59:23.506 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:23.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:279: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-21T01:59:23.577 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:59:27.110 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-21T01:59:27.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:284: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db2 2026-02-21T01:59:27.270 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:286: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-21T01:59:31.210 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:31.263 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:289: TEST_bluestore: ceph-objectstore-tool --type bluestore --data-path td/osd-bluefs-volume-ops/2 --op fsck --no-mon-config 2026-02-21T01:59:35.198 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:35.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:292: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-21T01:59:35.872 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records in 2026-02-21T01:59:35.872 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records out 2026-02-21T01:59:35.872 INFO:tasks.workunit.client.0.trial009.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.613345 s, 1.8 GB/s 2026-02-21T01:59:35.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:293: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-21T01:59:35.889 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:59:36.006 INFO:tasks.workunit.client.0.trial009.stderr:Unable to migrate Slow volume to new location, please allocate new DB or WAL with --bluefs-bdev-new-db(wal) command 2026-02-21T01:59:36.008 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:299: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-21T01:59:36.009 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:302: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:59:39.810 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:39.865 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:305: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-21T01:59:41.218 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records in 2026-02-21T01:59:41.219 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records out 2026-02-21T01:59:41.219 INFO:tasks.workunit.client.0.trial009.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.19931 s, 895 MB/s 2026-02-21T01:59:41.219 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:307: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --devs-source td/osd-bluefs-volume-ops/2/block.db --devs-source td/osd-bluefs-volume-ops/2/block.wal --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-21T01:59:41.232 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:59:44.782 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-21T01:59:44.782 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-21T01:59:44.783 INFO:tasks.workunit.client.0.trial009.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/2/db2 2026-02-21T01:59:44.814 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:314: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-21T01:59:48.546 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:48.593 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:317: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/3/wal2 count=1024 bs=1M 2026-02-21T01:59:49.200 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records in 2026-02-21T01:59:49.200 INFO:tasks.workunit.client.0.trial009.stderr:1024+0 records out 2026-02-21T01:59:49.200 INFO:tasks.workunit.client.0.trial009.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.605146 s, 1.8 GB/s 2026-02-21T01:59:49.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:319: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/wal2 --command bluefs-bdev-migrate 2026-02-21T01:59:49.216 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T01:59:53.354 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block 2026-02-21T01:59:53.354 INFO:tasks.workunit.client.0.trial009.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-21T01:59:53.354 INFO:tasks.workunit.client.0.trial009.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/3/wal2 2026-02-21T01:59:53.386 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:325: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-21T01:59:57.178 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T01:59:57.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:327: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-21T01:59:57.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T01:59:57.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T01:59:57.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T01:59:57.230 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T01:59:57.231 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T01:59:57.232 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T01:59:57.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T01:59:57.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T01:59:57.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T01:59:57.233 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T01:59:57.234 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T01:59:57.234 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T01:59:57.235 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-21T01:59:57.235 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T01:59:57.235 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T01:59:57.235 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T01:59:57.235 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T01:59:57.253 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:59:57.245+0000 7fac171d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:59:57.255 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:59:57.249+0000 7fac171d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:59:57.256 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:59:57.249+0000 7fac171d6a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T01:59:57.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T01:59:57.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T01:59:57.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T01:59:57.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T01:59:57.450 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T01:59:57.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:59:57.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T01:59:57.451 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T01:59:57.451 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:59:57.452 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:59:57.655 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:59:57.707 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T01:59:57.701+0000 7fac171d6a40 -1 Falling back to public interface 2026-02-21T01:59:58.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:59:58.657 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:59:58.657 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T01:59:58.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T01:59:58.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:59:58.658 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T01:59:58.870 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T01:59:59.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T01:59:59.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T01:59:59.872 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T01:59:59.873 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T01:59:59.873 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T01:59:59.873 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:00:00.084 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:01.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:01.085 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:01.086 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T02:00:01.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T02:00:01.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:01.087 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:00:01.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:02.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:02.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:02.292 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T02:00:02.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T02:00:02.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:02.292 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:00:02.497 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:03.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:03.498 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:03.498 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T02:00:03.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T02:00:03.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:03.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:00:03.708 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:04.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:04.709 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:04.709 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T02:00:04.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T02:00:04.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:04.710 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:00:04.917 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:05.915 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:05.909+0000 7fac171d6a40 -1 osd.0 64 log_to_monitors true 2026-02-21T02:00:05.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:05.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:05.919 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T02:00:05.919 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T02:00:05.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:05.920 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:00:06.126 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 72 up_thru 62 down_at 64 last_clean_interval [53,64) [v2:127.0.0.1:6802/1485575987,v1:127.0.0.1:6803/1485575987] [v2:127.0.0.1:6804/1485575987,v1:127.0.0.1:6805/1485575987] exists,up 52e40f1b-ea5c-4ac8-9da8-b96001b82630 2026-02-21T02:00:06.126 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:00:06.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:00:06.127 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:00:06.127 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T02:00:06.129 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: osd_pid0=128033 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:329: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:00:06.130 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:00:06.131 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T02:00:06.133 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-21T02:00:06.134 INFO:tasks.workunit.client.0.trial009.stdout:start osd.1 2026-02-21T02:00:06.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-21T02:00:06.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:00:06.135 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-21T02:00:06.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-21T02:00:06.135 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T02:00:06.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T02:00:06.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T02:00:06.155 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:06.149+0000 7fe038e65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:06.156 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:06.149+0000 7fe038e65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:06.158 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:06.153+0000 7fe038e65a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:06.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-21T02:00:06.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:00:06.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-21T02:00:06.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:00:06.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:00:06.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:06.337 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:00:06.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:00:06.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:06.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:06.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:06.571 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:06.565+0000 7fe038e65a40 -1 Falling back to public interface 2026-02-21T02:00:07.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:07.554 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:07.554 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:00:07.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:00:07.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:07.555 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:07.761 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:08.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:08.763 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:08.763 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:00:08.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:00:08.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:08.764 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:08.969 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:09.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:09.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:09.972 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T02:00:09.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T02:00:09.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:09.973 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:10.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:11.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:11.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:11.184 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T02:00:11.184 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T02:00:11.185 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:11.185 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:11.395 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:12.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:12.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:12.397 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T02:00:12.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T02:00:12.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:12.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:12.601 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:13.602 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:13.603 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:13.603 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T02:00:13.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T02:00:13.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:13.604 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:13.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:14.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:14.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:14.807 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T02:00:14.807 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T02:00:14.807 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:14.807 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-21T02:00:14.839 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:14.833+0000 7fe038e65a40 -1 osd.1 65 log_to_monitors true 2026-02-21T02:00:15.031 INFO:tasks.workunit.client.0.trial009.stdout:osd.1 up in weight 1 up_from 75 up_thru 64 down_at 66 last_clean_interval [56,65) [v2:127.0.0.1:6810/1422019519,v1:127.0.0.1:6811/1422019519] [v2:127.0.0.1:6812/1422019519,v1:127.0.0.1:6813/1422019519] exists,up 78fc2c9f-9040-4ea9-9180-d6f41d663f2f 2026-02-21T02:00:15.032 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:00:15.032 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:00:15.032 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:00:15.032 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: osd_pid1=129104 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:331: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:00:15.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:00:15.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:00:15.037 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T02:00:15.038 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-21T02:00:15.039 INFO:tasks.workunit.client.0.trial009.stdout:start osd.2 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T02:00:15.040 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T02:00:15.060 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:15.053+0000 7fccf66fda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:15.061 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:15.053+0000 7fccf66fda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:15.062 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:15.057+0000 7fccf66fda40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:15.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:00:15.256 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:00:15.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:15.257 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:15.462 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:15.471 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:15.465+0000 7fccf66fda40 -1 Falling back to public interface 2026-02-21T02:00:16.464 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:16.464 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:16.464 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:00:16.464 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:00:16.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:16.465 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:16.675 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:17.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:17.677 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:17.677 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:00:17.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:00:17.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:17.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:17.891 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:18.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:18.893 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:18.893 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T02:00:18.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T02:00:18.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:18.894 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:19.109 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:20.111 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:20.111 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:20.112 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T02:00:20.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T02:00:20.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:20.112 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:20.330 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:21.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:21.331 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:21.332 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T02:00:21.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T02:00:21.332 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:21.333 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:21.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:22.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:22.552 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:22.552 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T02:00:22.553 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T02:00:22.553 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:22.553 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:22.718 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:22.713+0000 7fccf66fda40 -1 osd.2 68 log_to_monitors true 2026-02-21T02:00:22.782 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:23.783 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:23.783 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:23.784 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T02:00:23.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T02:00:23.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:23.784 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-21T02:00:23.992 INFO:tasks.workunit.client.0.trial009.stdout:osd.2 up in weight 1 up_from 78 up_thru 78 down_at 68 last_clean_interval [59,68) [v2:127.0.0.1:6818/3450536651,v1:127.0.0.1:6819/3450536651] [v2:127.0.0.1:6820/3450536651,v1:127.0.0.1:6821/3450536651] exists,up 130cca78-4d57-40fb-a845-e0ee751e63c8 2026-02-21T02:00:23.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:00:23.992 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:00:23.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:00:23.993 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: osd_pid2=130165 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:333: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:00:23.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:00:23.998 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T02:00:23.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-21T02:00:24.000 INFO:tasks.workunit.client.0.trial009.stdout:start osd.3 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T02:00:24.001 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T02:00:24.015 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:24.009+0000 7f094821ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:24.016 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:24.009+0000 7f094821ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:24.017 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:24.009+0000 7f094821ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:00:24.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-21T02:00:24.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:00:24.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-21T02:00:24.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:00:24.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:00:24.206 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:24.207 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:00:24.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:00:24.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:24.207 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:24.413 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:24.455 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:24.449+0000 7f094821ba40 -1 Falling back to public interface 2026-02-21T02:00:25.414 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:25.415 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:25.415 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:00:25.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:00:25.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:25.416 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:25.621 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:26.622 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:26.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:26.623 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:00:26.623 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:00:26.624 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:26.624 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:26.833 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:27.834 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:27.835 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:27.835 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T02:00:27.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T02:00:27.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:27.836 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:28.043 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:29.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:29.044 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:29.045 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T02:00:29.045 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T02:00:29.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:29.046 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:29.248 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:30.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:30.250 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:30.251 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T02:00:30.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T02:00:30.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:30.251 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:30.456 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:31.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:31.457 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:31.458 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T02:00:31.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T02:00:31.458 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:31.459 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:31.662 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:00:32.170 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:00:32.164+0000 7f094821ba40 -1 osd.3 69 log_to_monitors true 2026-02-21T02:00:32.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:00:32.664 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:00:32.664 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T02:00:32.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T02:00:32.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:00:32.665 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-21T02:00:32.879 INFO:tasks.workunit.client.0.trial009.stdout:osd.3 up in weight 1 up_from 81 up_thru 81 down_at 70 last_clean_interval [62,69) [v2:127.0.0.1:6826/2586461675,v1:127.0.0.1:6827/2586461675] [v2:127.0.0.1:6828/2586461675,v1:127.0.0.1:6829/2586461675] exists,up e1c504e5-b193-4b0a-a81c-ad143b0184ca 2026-02-21T02:00:32.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:00:32.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:00:32.880 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:00:32.880 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-21T02:00:32.881 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: osd_pid3=131232 2026-02-21T02:00:32.881 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:337: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-21T02:00:32.900 INFO:tasks.workunit.client.0.trial009.stdout:hints = 1 2026-02-21T02:00:32.900 INFO:tasks.workunit.client.0.trial009.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-21T02:00:32.900 INFO:tasks.workunit.client.0.trial009.stdout:Object prefix: benchmark_data_trial009_132283 2026-02-21T02:00:32.900 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T02:00:32.900 INFO:tasks.workunit.client.0.trial009.stdout: 0 0 0 0 0 0 - 0 2026-02-21T02:00:33.900 INFO:tasks.workunit.client.0.trial009.stdout: 1 15 10929 10914 42.6294 42.6328 0.0010014 0.00146306 2026-02-21T02:00:34.900 INFO:tasks.workunit.client.0.trial009.stdout: 2 15 21550 21535 42.0572 41.4883 0.00140363 0.00148401 2026-02-21T02:00:35.900 INFO:tasks.workunit.client.0.trial009.stdout: 3 16 32161 32145 41.8522 41.4453 0.00160109 0.00149144 2026-02-21T02:00:36.900 INFO:tasks.workunit.client.0.trial009.stdout: 4 16 38877 38861 37.9472 26.2344 0.114455 0.00163506 2026-02-21T02:00:37.900 INFO:tasks.workunit.client.0.trial009.stdout: 5 16 48215 48199 37.6526 36.4766 0.00121884 0.00165821 2026-02-21T02:00:38.900 INFO:tasks.workunit.client.0.trial009.stdout: 6 16 58771 58755 38.249 41.2344 0.00143819 0.00163225 2026-02-21T02:00:39.900 INFO:tasks.workunit.client.0.trial009.stdout: 7 16 69345 69329 38.6851 41.3047 0.00161069 0.00161392 2026-02-21T02:00:40.900 INFO:tasks.workunit.client.0.trial009.stdout: 8 16 79765 79749 38.937 40.7031 0.00107718 0.00160349 2026-02-21T02:00:41.900 INFO:tasks.workunit.client.0.trial009.stdout: 9 16 90376 90360 39.2157 41.4492 0.00128406 0.0015921 2026-02-21T02:00:42.900 INFO:tasks.workunit.client.0.trial009.stdout: 10 16 101104 101088 39.4845 41.9062 0.00163758 0.0015813 2026-02-21T02:00:43.901 INFO:tasks.workunit.client.0.trial009.stdout: 11 16 111821 111805 39.7004 41.8633 0.00113893 0.0015727 2026-02-21T02:00:44.901 INFO:tasks.workunit.client.0.trial009.stdout: 12 16 122430 122414 39.8452 41.4414 0.00158936 0.00156699 2026-02-21T02:00:45.901 INFO:tasks.workunit.client.0.trial009.stdout: 13 16 132996 132980 39.9548 41.2734 0.00115416 0.00156269 2026-02-21T02:00:46.901 INFO:tasks.workunit.client.0.trial009.stdout: 14 16 143700 143684 40.0872 41.8125 0.00126779 0.00155752 2026-02-21T02:00:47.901 INFO:tasks.workunit.client.0.trial009.stdout: 15 16 147799 147783 38.4821 16.0117 0.0200863 0.00162094 2026-02-21T02:00:48.901 INFO:tasks.workunit.client.0.trial009.stdout: 16 16 153006 152990 37.3481 20.3398 0.00153853 0.00167189 2026-02-21T02:00:49.901 INFO:tasks.workunit.client.0.trial009.stdout: 17 16 163251 163235 37.505 40.0195 0.00195006 0.00166487 2026-02-21T02:00:50.901 INFO:tasks.workunit.client.0.trial009.stdout: 18 16 173201 173185 37.5805 38.8672 0.00100432 0.00166154 2026-02-21T02:00:51.901 INFO:tasks.workunit.client.0.trial009.stdout: 19 16 183312 183296 37.6812 39.4961 0.00144674 0.0016571 2026-02-21T02:00:52.901 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T02:00:52.899009+0000 min lat: 0.000500114 max lat: 0.286263 avg lat: 0.00165333 2026-02-21T02:00:52.901 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T02:00:52.902 INFO:tasks.workunit.client.0.trial009.stdout: 20 16 193400 193384 37.7673 39.4062 0.00184827 0.00165333 2026-02-21T02:00:53.901 INFO:tasks.workunit.client.0.trial009.stdout: 21 16 203597 203581 37.8655 39.832 0.00179763 0.00164903 2026-02-21T02:00:54.901 INFO:tasks.workunit.client.0.trial009.stdout: 22 16 213590 213574 37.9185 39.0352 0.00131768 0.00164673 2026-02-21T02:00:55.901 INFO:tasks.workunit.client.0.trial009.stdout: 23 16 223728 223712 37.9915 39.6016 0.00152908 0.00164356 2026-02-21T02:00:56.901 INFO:tasks.workunit.client.0.trial009.stdout: 24 16 233954 233938 38.0728 39.9453 0.00121507 0.00164004 2026-02-21T02:00:57.902 INFO:tasks.workunit.client.0.trial009.stdout: 25 15 243967 243952 38.1144 39.1172 0.00117163 0.00163829 2026-02-21T02:00:58.902 INFO:tasks.workunit.client.0.trial009.stdout: 26 16 252898 252882 37.99 34.8828 0.0737311 0.00164079 2026-02-21T02:00:59.902 INFO:tasks.workunit.client.0.trial009.stdout: 27 16 253005 252989 36.5984 0.417969 0.182472 0.00170331 2026-02-21T02:01:00.902 INFO:tasks.workunit.client.0.trial009.stdout: 28 16 253091 253075 35.3033 0.335938 0.200426 0.00176293 2026-02-21T02:01:01.902 INFO:tasks.workunit.client.0.trial009.stdout: 29 16 258989 258973 34.8804 23.0391 0.00114683 0.00179028 2026-02-21T02:01:02.902 INFO:tasks.workunit.client.0.trial009.stdout:Total time run: 30.0012 2026-02-21T02:01:02.902 INFO:tasks.workunit.client.0.trial009.stdout:Total writes made: 269121 2026-02-21T02:01:02.902 INFO:tasks.workunit.client.0.trial009.stdout:Write size: 4096 2026-02-21T02:01:02.902 INFO:tasks.workunit.client.0.trial009.stdout:Object size: 4096 2026-02-21T02:01:02.902 INFO:tasks.workunit.client.0.trial009.stdout:Bandwidth (MB/sec): 35.0404 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Bandwidth: 11.7919 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Max bandwidth (MB/sec): 42.6328 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Min bandwidth (MB/sec): 0.335938 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Average IOPS: 8970 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Stddev IOPS: 3018.73 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Max IOPS: 10914 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Min IOPS: 86 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Average Latency(s): 0.00178214 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Latency(s): 0.00578831 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Max latency(s): 0.286263 2026-02-21T02:01:02.903 INFO:tasks.workunit.client.0.trial009.stdout:Min latency(s): 0.000500114 2026-02-21T02:01:02.905 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:339: TEST_bluestore: wait_for_clean 2026-02-21T02:01:02.905 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T02:01:02.906 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T02:01:02.906 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T02:01:02.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T02:01:02.907 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T02:01:02.907 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T02:01:02.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T02:01:02.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T02:01:02.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T02:01:02.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T02:01:02.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T02:01:02.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T02:01:02.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T02:01:02.996 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T02:01:02.996 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T02:01:03.199 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-21T02:01:03.199 INFO:tasks.workunit.client.0.trial009.stderr:1 2026-02-21T02:01:03.199 INFO:tasks.workunit.client.0.trial009.stderr:2 2026-02-21T02:01:03.200 INFO:tasks.workunit.client.0.trial009.stderr:3' 2026-02-21T02:01:03.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T02:01:03.200 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T02:01:03.200 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T02:01:03.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645326 2026-02-21T02:01:03.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645326 2026-02-21T02:01:03.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645326' 2026-02-21T02:01:03.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T02:01:03.279 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-21T02:01:03.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=322122547212 2026-02-21T02:01:03.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 322122547212 2026-02-21T02:01:03.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645326 1-322122547212' 2026-02-21T02:01:03.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T02:01:03.357 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-21T02:01:03.435 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=335007449099 2026-02-21T02:01:03.435 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 335007449099 2026-02-21T02:01:03.435 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645326 1-322122547212 2-335007449099' 2026-02-21T02:01:03.435 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T02:01:03.436 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-21T02:01:03.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=347892350985 2026-02-21T02:01:03.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 347892350985 2026-02-21T02:01:03.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-309237645326 1-322122547212 2-335007449099 3-347892350985' 2026-02-21T02:01:03.513 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T02:01:03.514 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-309237645326 2026-02-21T02:01:03.514 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T02:01:03.515 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T02:01:03.516 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-309237645326 2026-02-21T02:01:03.516 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T02:01:03.517 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645326 2026-02-21T02:01:03.518 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 309237645326 2026-02-21T02:01:03.518 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 309237645326' 2026-02-21T02:01:03.518 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:01:03.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645324 -lt 309237645326 2026-02-21T02:01:03.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T02:01:04.739 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T02:01:04.740 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:01:04.948 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645326 -lt 309237645326 2026-02-21T02:01:04.948 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T02:01:04.949 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-322122547212 2026-02-21T02:01:04.949 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T02:01:04.950 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-21T02:01:04.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-322122547212 2026-02-21T02:01:04.951 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T02:01:04.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=322122547212 2026-02-21T02:01:04.952 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.1 seq 322122547212 2026-02-21T02:01:04.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 322122547212' 2026-02-21T02:01:04.953 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-21T02:01:05.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 322122547212 -lt 322122547212 2026-02-21T02:01:05.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T02:01:05.179 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-335007449099 2026-02-21T02:01:05.179 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T02:01:05.180 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-21T02:01:05.181 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-335007449099 2026-02-21T02:01:05.181 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T02:01:05.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=335007449099 2026-02-21T02:01:05.183 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.2 seq 335007449099 2026-02-21T02:01:05.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 335007449099' 2026-02-21T02:01:05.183 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-21T02:01:05.401 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 335007449099 -lt 335007449099 2026-02-21T02:01:05.401 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T02:01:05.402 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-347892350985 2026-02-21T02:01:05.402 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T02:01:05.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-21T02:01:05.404 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-347892350985 2026-02-21T02:01:05.404 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T02:01:05.405 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=347892350985 2026-02-21T02:01:05.405 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.3 seq 347892350985 2026-02-21T02:01:05.406 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 347892350985' 2026-02-21T02:01:05.406 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-21T02:01:05.624 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 347892350985 -lt 347892350985 2026-02-21T02:01:05.625 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T02:01:05.625 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T02:01:05.625 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T02:01:05.906 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T02:01:05.906 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T02:01:05.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T02:01:05.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T02:01:05.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T02:01:05.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T02:01:05.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T02:01:05.907 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T02:01:06.118 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T02:01:06.118 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T02:01:06.119 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T02:01:06.119 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T02:01:06.404 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T02:01:06.405 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T02:01:06.405 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T02:01:06.405 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T02:01:06.713 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T02:01:06.714 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T02:01:06.714 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T02:01:06.715 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T02:01:06.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T02:01:06.716 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T02:01:06.717 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T02:01:06.718 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:06.718 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T02:01:06.719 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T02:01:06.719 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:06.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:01:06.721 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T02:01:06.721 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-21T02:01:12.975 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T02:01:12.976 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:12.976 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:12.976 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.87226 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bluefs-volume-ops 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:12.978 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T02:01:12.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-21T02:01:12.979 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T02:01:12.979 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T02:01:12.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T02:01:12.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T02:01:12.979 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T02:01:12.980 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T02:01:12.981 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T02:01:12.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T02:01:12.982 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T02:01:12.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T02:01:12.983 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T02:01:12.984 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T02:01:12.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:12.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T02:01:12.985 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T02:01:12.986 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:12.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:01:12.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T02:01:12.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-21T02:01:12.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T02:01:12.989 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:12.990 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:12.990 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.87226 2026-02-21T02:01:12.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T02:01:12.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T02:01:12.991 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-21T02:01:12.992 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T02:01:12.992 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:12.993 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:12.993 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.87226 2026-02-21T02:01:12.994 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T02:01:12.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T02:01:12.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T02:01:12.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-21T02:01:12.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore2 td/osd-bluefs-volume-ops 2026-02-21T02:01:12.995 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:343: TEST_bluestore2: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:12.996 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: ulimit -n 2026-02-21T02:01:12.996 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: local flimit=4096 2026-02-21T02:01:12.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:346: TEST_bluestore2: '[' 4096 -lt 1536 ']' 2026-02-21T02:01:12.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: export CEPH_MON=127.0.0.1:7146 2026-02-21T02:01:12.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: CEPH_MON=127.0.0.1:7146 2026-02-21T02:01:12.997 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:350: TEST_bluestore2: export CEPH_ARGS 2026-02-21T02:01:12.997 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: uuidgen 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: CEPH_ARGS+='--fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:352: TEST_bluestore2: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:353: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:354: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:355: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:356: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:357: TEST_bluestore2: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:358: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:359: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:360: TEST_bluestore2: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-21T02:01:12.999 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:362: TEST_bluestore2: run_mon td/osd-bluefs-volume-ops a 2026-02-21T02:01:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T02:01:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T02:01:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T02:01:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-21T02:01:13.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:13.022 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T02:01:13.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:13.023 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:13.024 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:13.024 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:13.024 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:13.024 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:13.024 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T02:01:13.053 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T02:01:13.054 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T02:01:13.054 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T02:01:13.054 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T02:01:13.054 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T02:01:13.055 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T02:01:13.055 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T02:01:13.055 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.87226/ceph-mon.a.asok 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T02:01:13.056 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.87226/ceph-mon.a.asok config get fsid 2026-02-21T02:01:13.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T02:01:13.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T02:01:13.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T02:01:13.113 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T02:01:13.114 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.87226/ceph-mon.a.asok 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T02:01:13.115 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.87226/ceph-mon.a.asok config get mon_host 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:363: TEST_bluestore2: run_mgr td/osd-bluefs-volume-ops x 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-21T02:01:13.171 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:13.290 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:13.291 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T02:01:13.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:364: TEST_bluestore2: run_osd td/osd-bluefs-volume-ops 0 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:01:13.314 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:13.315 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T02:01:13.316 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T02:01:13.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T02:01:13.317 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T02:01:13.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7f578440-837e-44ad-9a37-8303dbd53edf 2026-02-21T02:01:13.317 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7f578440-837e-44ad-9a37-8303dbd53edf' 2026-02-21T02:01:13.318 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 7f578440-837e-44ad-9a37-8303dbd53edf 2026-02-21T02:01:13.318 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T02:01:13.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDpEZlpvk1eExAAs2icwQ47rwsDNTZOi1ohwA== 2026-02-21T02:01:13.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDpEZlpvk1eExAAs2icwQ47rwsDNTZOi1ohwA=="}' 2026-02-21T02:01:13.328 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7f578440-837e-44ad-9a37-8303dbd53edf -i td/osd-bluefs-volume-ops/0/new.json 2026-02-21T02:01:13.440 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:01:13.448 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-21T02:01:13.449 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDpEZlpvk1eExAAs2icwQ47rwsDNTZOi1ohwA== --osd-uuid 7f578440-837e-44ad-9a37-8303dbd53edf 2026-02-21T02:01:13.468 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:13.460+0000 7eff03f15a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:13.469 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:13.464+0000 7eff03f15a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:13.470 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:13.464+0000 7eff03f15a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:13.471 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:13.464+0000 7eff03f15a40 -1 bdev(0x55b966444c00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-21T02:01:13.471 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:13.464+0000 7eff03f15a40 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-21T02:01:14.781 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-21T02:01:14.781 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T02:01:14.782 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T02:01:14.782 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T02:01:14.782 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T02:01:14.905 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T02:01:14.906 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T02:01:14.906 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:01:14.907 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T02:01:14.907 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T02:01:14.907 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T02:01:14.932 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:14.924+0000 7fadbb9b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:14.934 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:14.928+0000 7fadbb9b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:14.935 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:14.928+0000 7fadbb9b5a40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:15.022 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:01:15.023 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:01:15.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:15.023 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:15.136 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:01:15.550 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:15.544+0000 7fadbb9b5a40 -1 Falling back to public interface 2026-02-21T02:01:16.138 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:01:16.138 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:16.139 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:01:16.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:01:16.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:16.139 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:16.354 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:01:16.650 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:16.644+0000 7fadbb9b5a40 -1 osd.0 0 log_to_monitors true 2026-02-21T02:01:17.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:01:17.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:17.356 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:01:17.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:01:17.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:17.357 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:17.575 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6800/2590999114,v1:127.0.0.1:6801/2590999114] [v2:127.0.0.1:6802/2590999114,v1:127.0.0.1:6803/2590999114] exists,up 7f578440-837e-44ad-9a37-8303dbd53edf 2026-02-21T02:01:17.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:01:17.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:01:17.576 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:01:17.577 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T02:01:17.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: osd_pid0=134100 2026-02-21T02:01:17.577 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:367: TEST_bluestore2: sleep 5 2026-02-21T02:01:22.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:368: TEST_bluestore2: create_pool foo 16 2026-02-21T02:01:22.579 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-21T02:01:22.849 INFO:tasks.workunit.client.0.trial009.stderr:pool 'foo' already exists 2026-02-21T02:01:22.861 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T02:01:23.862 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:370: TEST_bluestore2: retry = 0 2026-02-21T02:01:23.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 370: retry: command not found 2026-02-21T02:01:23.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:371: TEST_bluestore2: [[ '' -le 5 ]] 2026-02-21T02:01:23.863 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:373: TEST_bluestore2: timeout 60 rados bench -p foo 10 write --write-omap --no-cleanup 2026-02-21T02:01:23.884 INFO:tasks.workunit.client.0.trial009.stdout:hints = 1 2026-02-21T02:01:23.885 INFO:tasks.workunit.client.0.trial009.stdout:Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects 2026-02-21T02:01:23.885 INFO:tasks.workunit.client.0.trial009.stdout:Object prefix: benchmark_data_trial009_136928 2026-02-21T02:01:23.930 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T02:01:23.930 INFO:tasks.workunit.client.0.trial009.stdout: 0 0 0 0 0 0 - 0 2026-02-21T02:01:24.930 INFO:tasks.workunit.client.0.trial009.stdout: 1 16 187 171 683.933 684 0.0752283 0.0910475 2026-02-21T02:01:25.930 INFO:tasks.workunit.client.0.trial009.stdout: 2 16 371 355 709.934 736 0.0923683 0.0880283 2026-02-21T02:01:26.931 INFO:tasks.workunit.client.0.trial009.stdout: 3 16 555 539 718.6 736 0.081146 0.0879604 2026-02-21T02:01:27.931 INFO:tasks.workunit.client.0.trial009.stdout: 4 16 739 723 722.929 736 0.0962968 0.0878072 2026-02-21T02:01:28.931 INFO:tasks.workunit.client.0.trial009.stdout: 5 16 939 923 738.327 800 0.071099 0.0857652 2026-02-21T02:01:29.931 INFO:tasks.workunit.client.0.trial009.stdout: 6 16 1139 1123 748.594 800 0.0795509 0.085125 2026-02-21T02:01:30.931 INFO:tasks.workunit.client.0.trial009.stdout: 7 16 1299 1283 733.069 640 0.0981221 0.0866173 2026-02-21T02:01:31.931 INFO:tasks.workunit.client.0.trial009.stdout: 8 16 1491 1475 737.427 768 0.0934924 0.0860844 2026-02-21T02:01:32.931 INFO:tasks.workunit.client.0.trial009.stdout: 9 16 1699 1683 747.927 832 0.0681111 0.085102 2026-02-21T02:01:33.931 INFO:tasks.workunit.client.0.trial009.stdout: 10 16 1891 1875 749.927 768 0.0812708 0.0850984 2026-02-21T02:01:34.931 INFO:tasks.workunit.client.0.trial009.stdout:Total time run: 10.0699 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Total writes made: 1891 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Write size: 4194304 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Object size: 4194304 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Bandwidth (MB/sec): 751.147 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Bandwidth: 57.1703 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Max bandwidth (MB/sec): 832 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Min bandwidth (MB/sec): 640 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Average IOPS: 187 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Stddev IOPS: 14.2926 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Max IOPS: 208 2026-02-21T02:01:34.932 INFO:tasks.workunit.client.0.trial009.stdout:Min IOPS: 160 2026-02-21T02:01:34.933 INFO:tasks.workunit.client.0.trial009.stdout:Average Latency(s): 0.0850351 2026-02-21T02:01:34.933 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Latency(s): 0.0113361 2026-02-21T02:01:34.933 INFO:tasks.workunit.client.0.trial009.stdout:Max latency(s): 0.135915 2026-02-21T02:01:34.933 INFO:tasks.workunit.client.0.trial009.stdout:Min latency(s): 0.0353724 2026-02-21T02:01:34.943 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:376: TEST_bluestore2: sleep 10 2026-02-21T02:01:44.947 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-21T02:01:44.947 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: jq .bluefs.db_used_bytes 2026-02-21T02:01:45.033 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: db_used=535822336 2026-02-21T02:01:45.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-21T02:01:45.034 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: jq .bluefs.slow_used_bytes 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: spilled_over=255262720 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:380: TEST_bluestore2: (( retry+=1 )) 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: test 255262720 -eq 0 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: break 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:383: TEST_bluestore2: test 255262720 -gt 0 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 134100 2026-02-21T02:01:45.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: sleep 1 2026-02-21T02:01:46.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 134100 2026-02-21T02:01:46.122 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 385: kill: (134100) - No such process 2026-02-21T02:01:46.123 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:386: TEST_bluestore2: ceph osd down 0 2026-02-21T02:01:46.337 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T02:01:46.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:388: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/block --command bluefs-bdev-migrate 2026-02-21T02:01:46.361 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T02:01:47.785 INFO:tasks.workunit.client.0.trial009.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-21T02:01:47.806 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:393: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command bluefs-bdev-sizes 2026-02-21T02:01:47.826 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T02:01:48.277 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x100000000(4 GiB) : using 0x2ed8a000(750 MiB) 2026-02-21T02:01:48.369 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:396: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command fsck 2026-02-21T02:01:50.097 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T02:01:50.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:399: TEST_bluestore2: activate_osd td/osd-bluefs-volume-ops 0 2026-02-21T02:01:50.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:50.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T02:01:50.176 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:01:50.177 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:01:50.178 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:01:50.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:01:50.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T02:01:50.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T02:01:50.179 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T02:01:50.179 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T02:01:50.180 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T02:01:50.180 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:01:50.180 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-21T02:01:50.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T02:01:50.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T02:01:50.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T02:01:50.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T02:01:50.201 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:50.196+0000 7f6e2e42da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:50.203 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:50.196+0000 7f6e2e42da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:50.204 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:50.196+0000 7f6e2e42da40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:50.397 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T02:01:50.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:01:50.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T02:01:50.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:01:50.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:01:50.398 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:50.398 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:01:50.399 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:01:50.399 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:50.399 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:50.611 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:01:50.682 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:50.676+0000 7f6e2e42da40 -1 Falling back to public interface 2026-02-21T02:01:51.613 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:01:51.613 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:51.613 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:01:51.614 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:01:51.614 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:51.614 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:51.822 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:01:52.823 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:01:52.823 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:52.823 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:01:52.824 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:01:52.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:52.824 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:53.028 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:01:53.490 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:53.484+0000 7f6e2e42da40 -1 osd.0 10 log_to_monitors true 2026-02-21T02:01:54.030 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:01:54.030 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:54.030 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T02:01:54.031 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T02:01:54.031 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:54.031 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:54.247 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 13 up_thru 13 down_at 10 last_clean_interval [5,10) [v2:127.0.0.1:6800/3310040316,v1:127.0.0.1:6801/3310040316] [v2:127.0.0.1:6802/3310040316,v1:127.0.0.1:6803/3310040316] exists,up 7f578440-837e-44ad-9a37-8303dbd53edf 2026-02-21T02:01:54.248 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:01:54.248 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:01:54.248 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:01:54.248 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T02:01:54.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: osd_pid0=138159 2026-02-21T02:01:54.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:402: TEST_bluestore2: wait_for_clean 2026-02-21T02:01:54.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T02:01:54.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T02:01:54.249 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T02:01:54.250 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T02:01:54.250 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T02:01:54.250 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T02:01:54.250 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T02:01:54.251 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T02:01:54.251 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T02:01:54.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T02:01:54.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T02:01:54.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T02:01:54.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T02:01:54.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T02:01:54.345 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T02:01:54.566 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-21T02:01:54.566 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T02:01:54.566 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T02:01:54.566 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T02:01:54.651 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-21T02:01:54.651 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-21T02:01:54.651 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-55834574851' 2026-02-21T02:01:54.651 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T02:01:54.651 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-55834574851 2026-02-21T02:01:54.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T02:01:54.653 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T02:01:54.654 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-55834574851 2026-02-21T02:01:54.654 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T02:01:54.655 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574851 2026-02-21T02:01:54.655 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 55834574851' 2026-02-21T02:01:54.655 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 55834574851 2026-02-21T02:01:54.655 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:01:54.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 55834574851 2026-02-21T02:01:54.872 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T02:01:55.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T02:01:55.874 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:01:56.090 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-21T02:01:56.091 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T02:01:56.091 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T02:01:56.091 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T02:01:56.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T02:01:56.376 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T02:01:56.377 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T02:01:56.377 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T02:01:56.377 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T02:01:56.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T02:01:56.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T02:01:56.378 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T02:01:56.589 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T02:01:56.589 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T02:01:56.590 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T02:01:56.590 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T02:01:56.873 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T02:01:56.874 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T02:01:56.875 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T02:01:56.981 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T02:01:56.982 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T02:01:56.982 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T02:01:56.983 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T02:01:56.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T02:01:56.984 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T02:01:56.985 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T02:01:56.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:56.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T02:01:56.986 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T02:01:56.986 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:56.987 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:01:56.988 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T02:01:56.989 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-21T02:01:57.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T02:01:57.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.337 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.87226 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/osd-bluefs-volume-ops 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.339 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-21T02:01:57.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-21T02:01:57.340 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T02:01:57.340 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T02:01:57.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T02:01:57.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T02:01:57.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T02:01:57.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T02:01:57.342 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T02:01:57.342 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T02:01:57.343 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T02:01:57.343 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T02:01:57.344 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T02:01:57.344 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T02:01:57.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:57.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T02:01:57.345 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T02:01:57.346 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:01:57.347 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:01:57.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-21T02:01:57.348 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-21T02:01:57.349 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T02:01:57.349 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.349 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.350 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.87226 2026-02-21T02:01:57.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T02:01:57.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T02:01:57.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-21T02:01:57.352 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-21T02:01:57.352 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.352 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.353 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.87226 2026-02-21T02:01:57.354 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-21T02:01:57.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-21T02:01:57.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-21T02:01:57.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-21T02:01:57.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore_expand td/osd-bluefs-volume-ops 2026-02-21T02:01:57.355 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:406: TEST_bluestore_expand: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.355 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: ulimit -n 2026-02-21T02:01:57.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: local flimit=4096 2026-02-21T02:01:57.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:409: TEST_bluestore_expand: '[' 4096 -lt 1536 ']' 2026-02-21T02:01:57.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: export CEPH_MON=127.0.0.1:7146 2026-02-21T02:01:57.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: CEPH_MON=127.0.0.1:7146 2026-02-21T02:01:57.356 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:413: TEST_bluestore_expand: export CEPH_ARGS 2026-02-21T02:01:57.357 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: uuidgen 2026-02-21T02:01:57.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: CEPH_ARGS+='--fsid=6ea7bcc7-9fde-4bf2-ae20-b18d3891b38c --auth-supported=none ' 2026-02-21T02:01:57.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:415: TEST_bluestore_expand: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-21T02:01:57.358 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:416: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:417: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:418: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:419: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:420: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:421: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:422: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:423: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:425: TEST_bluestore_expand: run_mon td/osd-bluefs-volume-ops a 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-21T02:01:57.359 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-21T02:01:57.360 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.384 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-21T02:01:57.384 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:57.384 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:57.384 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:57.384 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.384 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.385 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:57.385 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-21T02:01:57.417 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-21T02:01:57.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-21T02:01:57.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T02:01:57.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T02:01:57.418 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-21T02:01:57.419 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.87226/ceph-mon.a.asok 2026-02-21T02:01:57.420 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T02:01:57.421 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.87226/ceph-mon.a.asok config get fsid 2026-02-21T02:01:57.476 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-21T02:01:57.476 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-21T02:01:57.476 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-21T02:01:57.476 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-21T02:01:57.477 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.478 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.87226/ceph-mon.a.asok 2026-02-21T02:01:57.479 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-21T02:01:57.479 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.87226/ceph-mon.a.asok config get mon_host 2026-02-21T02:01:57.533 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:426: TEST_bluestore_expand: run_mgr td/osd-bluefs-volume-ops x 2026-02-21T02:01:57.533 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-21T02:01:57.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-21T02:01:57.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-21T02:01:57.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-21T02:01:57.534 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-21T02:01:57.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-21T02:01:57.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:57.652 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:57.653 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:57.653 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.653 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.653 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:57.654 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T02:01:57.654 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-21T02:01:57.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:427: TEST_bluestore_expand: run_osd td/osd-bluefs-volume-ops 0 2026-02-21T02:01:57.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=6ea7bcc7-9fde-4bf2-ae20-b18d3891b38c --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T02:01:57.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-21T02:01:57.680 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-21T02:01:57.681 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T02:01:57.682 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-21T02:01:57.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=02a4cff4-0a1c-426d-b20e-ef432d6eda0a 2026-02-21T02:01:57.682 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 02a4cff4-0a1c-426d-b20e-ef432d6eda0a' 2026-02-21T02:01:57.682 INFO:tasks.workunit.client.0.trial009.stdout:add osd0 02a4cff4-0a1c-426d-b20e-ef432d6eda0a 2026-02-21T02:01:57.683 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-21T02:01:57.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAVEplpC1ROKRAAFRIJCtyu7lTuvktfT4I6zg== 2026-02-21T02:01:57.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAVEplpC1ROKRAAFRIJCtyu7lTuvktfT4I6zg=="}' 2026-02-21T02:01:57.696 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 02a4cff4-0a1c-426d-b20e-ef432d6eda0a -i td/osd-bluefs-volume-ops/0/new.json 2026-02-21T02:01:57.808 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:01:57.815 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-21T02:01:57.817 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=6ea7bcc7-9fde-4bf2-ae20-b18d3891b38c --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAVEplpC1ROKRAAFRIJCtyu7lTuvktfT4I6zg== --osd-uuid 02a4cff4-0a1c-426d-b20e-ef432d6eda0a 2026-02-21T02:01:57.838 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:57.832+0000 7fd59a70ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:57.839 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:57.832+0000 7fd59a70ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:57.840 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:57.832+0000 7fd59a70ba40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:57.841 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:57.836+0000 7fd59a70ba40 -1 bdev(0x562f8628cc00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-21T02:01:57.841 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:57.836+0000 7fd59a70ba40 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-21T02:01:59.548 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-21T02:01:59.549 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-21T02:01:59.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-21T02:01:59.550 INFO:tasks.workunit.client.0.trial009.stdout:adding osd0 key to auth repository 2026-02-21T02:01:59.550 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-21T02:01:59.677 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T02:01:59.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-21T02:01:59.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=6ea7bcc7-9fde-4bf2-ae20-b18d3891b38c --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:01:59.678 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-21T02:01:59.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-21T02:01:59.679 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-21T02:01:59.709 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:59.704+0000 7f64a624ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:59.711 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:59.704+0000 7f64a624ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:59.712 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:01:59.704+0000 7f64a624ca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:01:59.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-21T02:01:59.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:01:59.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T02:01:59.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:01:59.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:01:59.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:01:59.804 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:01:59.804 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:01:59.805 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:01:59.808 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:01:59.952 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:02:00.218 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:02:00.212+0000 7f64a624ca40 -1 Falling back to public interface 2026-02-21T02:02:00.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:02:00.953 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:02:00.954 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:02:00.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:02:00.954 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:02:00.955 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:02:01.172 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:02:01.330 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:02:01.324+0000 7f64a624ca40 -1 osd.0 0 log_to_monitors true 2026-02-21T02:02:02.173 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:02:02.174 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:02:02.174 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:02:02.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:02:02.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:02:02.175 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:02:02.392 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2738288091,v1:127.0.0.1:6803/2738288091] [v2:127.0.0.1:6804/2738288091,v1:127.0.0.1:6805/2738288091] exists,up 02a4cff4-0a1c-426d-b20e-ef432d6eda0a 2026-02-21T02:02:02.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:02:02.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:02:02.393 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:02:02.393 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T02:02:02.394 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: osd_pid0=140717 2026-02-21T02:02:02.394 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:430: TEST_bluestore_expand: sleep 5 2026-02-21T02:02:07.396 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:431: TEST_bluestore_expand: create_pool foo 16 2026-02-21T02:02:07.396 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create foo 16 2026-02-21T02:02:07.656 INFO:tasks.workunit.client.0.trial009.stderr:pool 'foo' already exists 2026-02-21T02:02:07.666 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-21T02:02:08.667 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:434: TEST_bluestore_expand: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-21T02:02:08.691 INFO:tasks.workunit.client.0.trial009.stdout:hints = 1 2026-02-21T02:02:08.691 INFO:tasks.workunit.client.0.trial009.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-21T02:02:08.691 INFO:tasks.workunit.client.0.trial009.stdout:Object prefix: benchmark_data_trial009_142468 2026-02-21T02:02:08.692 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T02:02:08.692 INFO:tasks.workunit.client.0.trial009.stdout: 0 0 0 0 0 0 - 0 2026-02-21T02:02:09.691 INFO:tasks.workunit.client.0.trial009.stdout: 1 15 16152 16137 63.0297 63.0352 0.00111624 0.000990026 2026-02-21T02:02:10.691 INFO:tasks.workunit.client.0.trial009.stdout: 2 16 32407 32391 63.2585 63.4922 0.00164817 0.000986816 2026-02-21T02:02:11.692 INFO:tasks.workunit.client.0.trial009.stdout: 3 16 48514 48498 63.1434 62.918 0.00113774 0.000988725 2026-02-21T02:02:12.692 INFO:tasks.workunit.client.0.trial009.stdout: 4 15 64740 64725 63.2029 63.3867 0.00059498 0.000987827 2026-02-21T02:02:13.692 INFO:tasks.workunit.client.0.trial009.stdout: 5 16 80775 80759 63.088 62.6328 0.000783181 0.000989689 2026-02-21T02:02:14.692 INFO:tasks.workunit.client.0.trial009.stdout: 6 15 97022 97007 63.1506 63.4688 0.00095416 0.000988725 2026-02-21T02:02:15.692 INFO:tasks.workunit.client.0.trial009.stdout: 7 16 112439 112423 62.7311 60.2188 0.00115178 0.000995324 2026-02-21T02:02:16.692 INFO:tasks.workunit.client.0.trial009.stdout: 8 16 127138 127122 62.0665 57.418 0.00111544 0.00100601 2026-02-21T02:02:17.692 INFO:tasks.workunit.client.0.trial009.stdout: 9 16 141820 141804 61.5422 57.3516 0.00108282 0.00101459 2026-02-21T02:02:18.692 INFO:tasks.workunit.client.0.trial009.stdout: 10 16 156402 156386 61.0836 56.9609 0.00123358 0.00102222 2026-02-21T02:02:19.692 INFO:tasks.workunit.client.0.trial009.stdout: 11 16 170483 170467 60.5305 55.0039 0.00105614 0.00103157 2026-02-21T02:02:20.692 INFO:tasks.workunit.client.0.trial009.stdout: 12 16 184443 184427 60.0302 54.5312 0.00103893 0.00104016 2026-02-21T02:02:21.692 INFO:tasks.workunit.client.0.trial009.stdout: 13 16 198404 198388 59.6072 54.5352 0.00236497 0.00104755 2026-02-21T02:02:22.692 INFO:tasks.workunit.client.0.trial009.stdout: 14 16 212476 212460 59.2756 54.9688 0.00115682 0.00105343 2026-02-21T02:02:23.692 INFO:tasks.workunit.client.0.trial009.stdout: 15 16 226160 226144 58.8872 53.4531 0.000667575 0.00106038 2026-02-21T02:02:24.692 INFO:tasks.workunit.client.0.trial009.stdout: 16 16 239900 239884 58.561 53.6719 0.00176981 0.00106629 2026-02-21T02:02:25.692 INFO:tasks.workunit.client.0.trial009.stdout: 17 16 253797 253781 58.3093 54.2852 0.00105861 0.00107091 2026-02-21T02:02:26.692 INFO:tasks.workunit.client.0.trial009.stdout: 18 16 267729 267713 58.0931 54.4219 0.000990547 0.00107489 2026-02-21T02:02:27.693 INFO:tasks.workunit.client.0.trial009.stdout: 19 16 281512 281496 57.869 53.8398 0.00117506 0.00107906 2026-02-21T02:02:28.693 INFO:tasks.workunit.client.0.trial009.stdout:2026-02-21T02:02:28.691199+0000 min lat: 0.000322211 max lat: 0.00530085 avg lat: 0.00108263 2026-02-21T02:02:28.693 INFO:tasks.workunit.client.0.trial009.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-21T02:02:28.693 INFO:tasks.workunit.client.0.trial009.stdout: 20 16 295352 295336 57.6785 54.0625 0.000876755 0.00108263 2026-02-21T02:02:29.693 INFO:tasks.workunit.client.0.trial009.stdout: 21 16 309142 309126 57.4968 53.8672 0.00106233 0.00108605 2026-02-21T02:02:30.693 INFO:tasks.workunit.client.0.trial009.stdout: 22 16 316048 316032 56.1094 26.9766 0.0113691 0.00111206 2026-02-21T02:02:31.693 INFO:tasks.workunit.client.0.trial009.stdout: 23 16 316645 316629 53.7712 2.33203 0.0228952 0.00116085 2026-02-21T02:02:32.693 INFO:tasks.workunit.client.0.trial009.stdout: 24 16 317260 317244 51.6309 2.40234 0.0239758 0.00120926 2026-02-21T02:02:33.693 INFO:tasks.workunit.client.0.trial009.stdout: 25 16 317849 317833 49.6577 2.30078 0.132914 0.00125221 2026-02-21T02:02:34.693 INFO:tasks.workunit.client.0.trial009.stdout: 26 16 317945 317929 47.7622 0.375 0.108159 0.00130363 2026-02-21T02:02:35.693 INFO:tasks.workunit.client.0.trial009.stdout: 27 16 320950 320934 46.4279 11.7383 0.00161901 0.00134521 2026-02-21T02:02:36.693 INFO:tasks.workunit.client.0.trial009.stdout: 28 16 334506 334490 46.6608 52.9531 0.000946766 0.00133849 2026-02-21T02:02:37.693 INFO:tasks.workunit.client.0.trial009.stdout: 29 16 348241 348225 46.9018 53.6523 0.000907392 0.00133158 2026-02-21T02:02:38.693 INFO:tasks.workunit.client.0.trial009.stdout:Total time run: 30.0011 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Total writes made: 361899 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Write size: 4096 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Object size: 4096 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Bandwidth (MB/sec): 47.1205 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Bandwidth: 21.1418 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Max bandwidth (MB/sec): 63.4922 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Min bandwidth (MB/sec): 0.375 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Average IOPS: 12062 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Stddev IOPS: 5412.3 2026-02-21T02:02:38.694 INFO:tasks.workunit.client.0.trial009.stdout:Max IOPS: 16254 2026-02-21T02:02:38.695 INFO:tasks.workunit.client.0.trial009.stdout:Min IOPS: 96 2026-02-21T02:02:38.695 INFO:tasks.workunit.client.0.trial009.stdout:Average Latency(s): 0.00132543 2026-02-21T02:02:38.695 INFO:tasks.workunit.client.0.trial009.stdout:Stddev Latency(s): 0.00432751 2026-02-21T02:02:38.695 INFO:tasks.workunit.client.0.trial009.stdout:Max latency(s): 0.319604 2026-02-21T02:02:38.695 INFO:tasks.workunit.client.0.trial009.stdout:Min latency(s): 0.000322211 2026-02-21T02:02:38.697 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:435: TEST_bluestore_expand: sleep 5 2026-02-21T02:02:43.700 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-21T02:02:43.700 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-21T02:02:43.786 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: total_space_before=4294967296 2026-02-21T02:02:43.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-21T02:02:43.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-21T02:02:43.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep free 2026-02-21T02:02:43.787 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d: -f 2 2026-02-21T02:02:43.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d, -f 1 2026-02-21T02:02:43.788 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-21T02:02:43.869 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: free_space_before=2812530688 2026-02-21T02:02:43.869 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 140717 2026-02-21T02:02:43.869 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-21T02:02:44.870 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 140717 2026-02-21T02:02:44.870 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 441: kill: (140717) - No such process 2026-02-21T02:02:44.871 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:442: TEST_bluestore_expand: ceph osd down 0 2026-02-21T02:02:45.076 INFO:tasks.workunit.client.0.trial009.stderr:osd.0 is already down. 2026-02-21T02:02:45.086 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:445: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 allocmap 2026-02-21T02:02:45.110 INFO:tasks.workunit.client.0.trial009.stdout:allocmap bluestore.allocmap 2026-02-21T02:03:29.248 INFO:tasks.workunit.client.0.trial009.stdout:allocmap success 2026-02-21T02:03:29.286 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:448: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T02:04:16.100 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T02:04:16.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:450: TEST_bluestore_expand: requested_space=4294967296 2026-02-21T02:04:16.143 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:451: TEST_bluestore_expand: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-21T02:04:16.144 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:452: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-21T02:04:16.167 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T02:04:59.391 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0x28b00000(651 MiB) 2026-02-21T02:04:59.391 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0x100000000(4 GiB) : using 0x585c5000(1.4 GiB) 2026-02-21T02:04:59.391 INFO:tasks.workunit.client.0.trial009.stdout:Expanding DB/WAL... 2026-02-21T02:04:59.391 INFO:tasks.workunit.client.0.trial009.stdout:1 : nothing to do, skipped 2026-02-21T02:04:59.391 INFO:tasks.workunit.client.0.trial009.stdout:2 : nothing to do, skipped 2026-02-21T02:04:59.523 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:455: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-21T02:05:45.256 INFO:tasks.workunit.client.0.trial009.stdout:fsck success 2026-02-21T02:05:45.283 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:458: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 qfsck 2026-02-21T02:05:45.307 INFO:tasks.workunit.client.0.trial009.stdout:qfsck bluestore.quick-fsck 2026-02-21T02:06:28.880 INFO:tasks.workunit.client.0.trial009.stdout:qfsck success 2026-02-21T02:06:28.903 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:460: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-21T02:06:28.919 INFO:tasks.workunit.client.0.trial009.stdout:inferring bluefs devices from bluestore path 2026-02-21T02:07:11.144 INFO:tasks.workunit.client.0.trial009.stdout:1 : device size 0x40000000(1 GiB) : using 0x28b00000(651 MiB) 2026-02-21T02:07:11.144 INFO:tasks.workunit.client.0.trial009.stdout:2 : device size 0x100000000(4 GiB) : using 0x585c5000(1.4 GiB) 2026-02-21T02:07:11.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:462: TEST_bluestore_expand: activate_osd td/osd-bluefs-volume-ops 0 2026-02-21T02:07:11.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:07:11.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-21T02:07:11.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=0 2026-02-21T02:07:11.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-21T02:07:11.308 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=6ea7bcc7-9fde-4bf2-ae20-b18d3891b38c --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-21T02:07:11.309 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-21T02:07:11.310 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-21T02:07:11.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-21T02:07:11.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-21T02:07:11.311 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-21T02:07:11.311 INFO:tasks.workunit.client.0.trial009.stdout:start osd.0 2026-02-21T02:07:11.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.0 2026-02-21T02:07:11.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 0 --fsid=c7711515-668c-4d41-8362-7afba87494f1 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=55b65c4a-c2c8-45b2-8f86-1db378367ca9 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=6ea7bcc7-9fde-4bf2-ae20-b18d3891b38c --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.87226/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-21T02:07:11.312 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-21T02:07:11.312 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 0 = 0 ']' 2026-02-21T02:07:11.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-21T02:07:11.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-21T02:07:11.313 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-21T02:07:11.338 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:07:11.332+0000 7f95652cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:07:11.339 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:07:11.332+0000 7f95652cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:07:11.340 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:07:11.336+0000 7f95652cca40 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-21T02:07:11.528 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 0 2026-02-21T02:07:11.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-21T02:07:11.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-21T02:07:11.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-21T02:07:11.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-21T02:07:11.529 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:11.530 INFO:tasks.workunit.client.0.trial009.stdout:0 2026-02-21T02:07:11.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-21T02:07:11.530 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:11.531 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:11.738 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:11.813 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:07:11.808+0000 7f95652cca40 -1 Falling back to public interface 2026-02-21T02:07:12.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:12.740 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:12.740 INFO:tasks.workunit.client.0.trial009.stdout:1 2026-02-21T02:07:12.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-21T02:07:12.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:12.741 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:12.961 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:13.962 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:13.963 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:13.963 INFO:tasks.workunit.client.0.trial009.stdout:2 2026-02-21T02:07:13.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 2 2026-02-21T02:07:13.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:13.964 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:14.181 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:15.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:15.182 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:15.183 INFO:tasks.workunit.client.0.trial009.stdout:3 2026-02-21T02:07:15.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 3 2026-02-21T02:07:15.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:15.183 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:15.400 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:16.402 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:16.402 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:16.402 INFO:tasks.workunit.client.0.trial009.stdout:4 2026-02-21T02:07:16.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 4 2026-02-21T02:07:16.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:16.403 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:16.618 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:17.620 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:17.620 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:17.621 INFO:tasks.workunit.client.0.trial009.stdout:5 2026-02-21T02:07:17.621 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 5 2026-02-21T02:07:17.621 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:17.622 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:17.839 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:18.840 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:18.841 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:18.841 INFO:tasks.workunit.client.0.trial009.stdout:6 2026-02-21T02:07:18.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 6 2026-02-21T02:07:18.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:18.842 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:19.057 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:20.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:20.059 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:20.059 INFO:tasks.workunit.client.0.trial009.stdout:7 2026-02-21T02:07:20.060 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 7 2026-02-21T02:07:20.060 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:20.060 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:20.276 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:21.278 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:21.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:21.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 8 2026-02-21T02:07:21.279 INFO:tasks.workunit.client.0.trial009.stdout:8 2026-02-21T02:07:21.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:21.280 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:21.499 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:22.500 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:22.501 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:22.501 INFO:tasks.workunit.client.0.trial009.stdout:9 2026-02-21T02:07:22.502 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 9 2026-02-21T02:07:22.502 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:22.502 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:22.720 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-21T02:07:23.184 INFO:tasks.workunit.client.0.trial009.stderr:2026-02-21T02:07:23.180+0000 7f95652cca40 -1 osd.0 10 log_to_monitors true 2026-02-21T02:07:23.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-21T02:07:23.722 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-21T02:07:23.723 INFO:tasks.workunit.client.0.trial009.stdout:10 2026-02-21T02:07:23.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 10 2026-02-21T02:07:23.723 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-21T02:07:23.724 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-21T02:07:23.932 INFO:tasks.workunit.client.0.trial009.stdout:osd.0 up in weight 1 up_from 13 up_thru 13 down_at 10 last_clean_interval [5,10) [v2:127.0.0.1:6802/2243008425,v1:127.0.0.1:6803/2243008425] [v2:127.0.0.1:6804/2243008425,v1:127.0.0.1:6805/2243008425] exists,up 02a4cff4-0a1c-426d-b20e-ef432d6eda0a 2026-02-21T02:07:23.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-21T02:07:23.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-21T02:07:23.933 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-21T02:07:23.933 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-21T02:07:23.934 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: osd_pid0=144842 2026-02-21T02:07:23.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:465: TEST_bluestore_expand: wait_for_clean 2026-02-21T02:07:23.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-21T02:07:23.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-21T02:07:23.935 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-21T02:07:23.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-21T02:07:23.936 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-21T02:07:23.936 INFO:tasks.workunit.client.0.trial009.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-21T02:07:23.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-21T02:07:23.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-21T02:07:23.936 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-21T02:07:24.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-21T02:07:24.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-21T02:07:24.035 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-21T02:07:24.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-21T02:07:24.036 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-21T02:07:24.036 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-21T02:07:24.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids=0 2026-02-21T02:07:24.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-21T02:07:24.255 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-21T02:07:24.255 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-21T02:07:24.335 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-21T02:07:24.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-21T02:07:24.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-55834574851' 2026-02-21T02:07:24.336 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-21T02:07:24.336 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-55834574851 2026-02-21T02:07:24.337 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-21T02:07:24.338 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-21T02:07:24.338 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-55834574851 2026-02-21T02:07:24.338 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-21T02:07:24.340 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574851 2026-02-21T02:07:24.340 INFO:tasks.workunit.client.0.trial009.stdout:waiting osd.0 seq 55834574851 2026-02-21T02:07:24.341 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 55834574851' 2026-02-21T02:07:24.341 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:07:24.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 55834574851 2026-02-21T02:07:24.559 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T02:07:25.560 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-21T02:07:25.560 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:07:25.779 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 1 -lt 55834574851 2026-02-21T02:07:25.779 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-21T02:07:26.780 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-21T02:07:26.781 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-21T02:07:27.000 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-21T02:07:27.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-21T02:07:27.001 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T02:07:27.002 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T02:07:27.286 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-21T02:07:27.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-21T02:07:27.504 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-21T02:07:27.505 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-21T02:07:27.505 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-21T02:07:27.505 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-21T02:07:27.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-21T02:07:27.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-21T02:07:27.803 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-21T02:07:27.804 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-21T02:07:27.804 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-21T02:07:27.887 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: total_space_after=4294967296 2026-02-21T02:07:27.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-21T02:07:27.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-21T02:07:27.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep free 2026-02-21T02:07:27.888 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d: -f 2 2026-02-21T02:07:27.889 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d, -f 1 2026-02-21T02:07:27.889 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-21T02:07:27.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: free_space_after=2785226752 2026-02-21T02:07:27.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:470: TEST_bluestore_expand: '[' 4294967296 '!=' 4294967296 ']' 2026-02-21T02:07:27.970 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:476: TEST_bluestore_expand: total_space_added=0 2026-02-21T02:07:27.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:477: TEST_bluestore_expand: free_space_added=-27303936 2026-02-21T02:07:27.971 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:479: TEST_bluestore_expand: let 'new_used_space=(0 - -27303936)' 2026-02-21T02:07:27.971 INFO:tasks.workunit.client.0.trial009.stdout:27303936 2026-02-21T02:07:27.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:480: TEST_bluestore_expand: echo 27303936 2026-02-21T02:07:27.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:482: TEST_bluestore_expand: '[' 27303936 -gt 131072 ']' 2026-02-21T02:07:27.972 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:483: TEST_bluestore_expand: echo 'total_space_added = 0' 2026-02-21T02:07:27.972 INFO:tasks.workunit.client.0.trial009.stdout:total_space_added = 0 2026-02-21T02:07:27.972 INFO:tasks.workunit.client.0.trial009.stdout:free_space_added = -27303936 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:484: TEST_bluestore_expand: echo 'free_space_added = -27303936' 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:485: TEST_bluestore_expand: return 1 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: return 1 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2403: main: code=1 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/osd-bluefs-volume-ops 1 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=1 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-21T02:07:27.974 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-21T02:07:27.975 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-21T02:07:27.975 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-21T02:07:28.279 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-21T02:07:28.280 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-21T02:07:28.281 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-21T02:07:28.282 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-21T02:07:28.284 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-21T02:07:28.284 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-21T02:07:28.284 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-21T02:07:28.286 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:07:28.286 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-21T02:07:28.286 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-21T02:07:28.287 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T02:07:28.288 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:07:28.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-21T02:07:28.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:181: teardown: '[' -n '' ']' 2026-02-21T02:07:28.289 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:185: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-21T02:07:28.290 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:186: teardown: mv td/osd-bluefs-volume-ops/bluestore_tool.log td/osd-bluefs-volume-ops/mgr.x.log td/osd-bluefs-volume-ops/mon.a.log td/osd-bluefs-volume-ops/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-21T02:07:28.291 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-21T02:07:28.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-21T02:07:28.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-21T02:07:28.351 INFO:tasks.workunit.client.0.trial009.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.87226 2026-02-21T02:07:28.351 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.87226 2026-02-21T02:07:28.352 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-21T02:07:28.352 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-21T02:07:28.352 INFO:tasks.workunit.client.0.trial009.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 1 2026-02-21T02:07:28.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T02:07:28.353 INFO:tasks.workunit:Stopping ['osd'] on client.0... 2026-02-21T02:07:28.354 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-21T02:07:28.670 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_36d388f27052dd940bdbaf543bb05bdba0acc155/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_36d388f27052dd940bdbaf543bb05bdba0acc155/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_36d388f27052dd940bdbaf543bb05bdba0acc155/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial009 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh' 2026-02-21T02:07:28.674 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T02:07:28.688 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T02:07:28.688 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T02:07:28.791 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T02:07:28.802 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T02:07:28.802 DEBUG:teuthology.orchestra.run.trial009:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T02:07:28.853 INFO:teuthology.orchestra.run.trial009.stdout: remote refid st t when poll reach delay offset jitter 2026-02-21T02:07:28.853 INFO:teuthology.orchestra.run.trial009.stdout:============================================================================== 2026-02-21T02:07:28.853 INFO:teuthology.orchestra.run.trial009.stdout:+23.157.160.168 129.6.15.28 2 u 64 64 377 3.109 +0.238 1.142 2026-02-21T02:07:28.853 INFO:teuthology.orchestra.run.trial009.stdout:+LAX.CALTICK.NET 17.253.83.251 2 u 15 64 377 61.995 +0.189 7.671 2026-02-21T02:07:28.853 INFO:teuthology.orchestra.run.trial009.stdout:*216.144.228.179 129.6.15.30 2 u 9 64 377 8.561 -0.394 1.229 2026-02-21T02:07:28.853 INFO:teuthology.orchestra.run.trial009.stdout:-144.202.41.38 ( 99.28.14.242 2 u 1 64 377 34.213 -2.340 1.775 2026-02-21T02:07:28.854 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T02:07:28.865 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T02:07:28.865 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T02:07:28.880 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T02:07:28.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T02:07:28.906 INFO:teuthology.task.internal:Duration was 1597.248797 seconds 2026-02-21T02:07:28.907 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T02:07:28.921 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T02:07:28.921 DEBUG:teuthology.orchestra.run.trial009:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T02:07:28.945 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T02:07:28.945 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial009.front.sepia.ceph.com 2026-02-21T02:07:28.946 DEBUG:teuthology.orchestra.run.trial009:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T02:07:28.993 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T02:07:28.993 DEBUG:teuthology.orchestra.run.trial009:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T02:07:29.255 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T02:07:29.256 DEBUG:teuthology.orchestra.run.trial009:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T02:07:29.264 INFO:teuthology.orchestra.run.trial009.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T02:07:29.265 INFO:teuthology.orchestra.run.trial009.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T02:07:29.265 INFO:teuthology.orchestra.run.trial009.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gzgzip 2026-02-21T02:07:29.265 INFO:teuthology.orchestra.run.trial009.stderr: -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T02:07:29.265 INFO:teuthology.orchestra.run.trial009.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T02:07:29.330 INFO:teuthology.orchestra.run.trial009.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T02:07:29.332 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T02:07:29.345 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T02:07:29.345 DEBUG:teuthology.orchestra.run.trial009:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T02:07:29.390 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T02:07:29.410 DEBUG:teuthology.orchestra.run.trial009:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:07:29.446 INFO:teuthology.orchestra.run.trial009.stdout:kernel.core_pattern = core 2026-02-21T02:07:29.455 DEBUG:teuthology.orchestra.run.trial009:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T02:07:29.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T02:07:29.503 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T02:07:29.524 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T02:07:29.526 DEBUG:teuthology.misc:Transferring archived files from trial009:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-21_00:21:18-rados-wip-yuri3-testing-2026-02-19-1610-tentacle-distro-default-trial/62246/remote/trial009 2026-02-21T02:07:29.528 DEBUG:teuthology.orchestra.run.trial009:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T02:58:12.594 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T02:58:12.600 DEBUG:teuthology.orchestra.run.trial009:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T02:58:16.396 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T02:58:16.413 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T02:58:16.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T02:58:16.430 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T02:58:16.430 DEBUG:teuthology.orchestra.run.trial009:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T02:58:16.435 INFO:teuthology.orchestra.run.trial009.stdout: 1179687 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 21 02:58 /home/ubuntu/cephtest 2026-02-21T02:58:16.436 INFO:teuthology.orchestra.run.trial009.stdout: 1188712 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 21 01:43 /home/ubuntu/cephtest/mnt.0 2026-02-21T02:58:16.436 INFO:teuthology.orchestra.run.trial009.stdout: 1188713 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 21 01:54 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T02:58:16.436 INFO:teuthology.orchestra.run.trial009.stdout: 1204375 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 21 01:54 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T02:58:16.436 INFO:teuthology.orchestra.run.trial009.stdout: 1204376 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 21 02:07 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-21T02:58:16.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T02:58:16.437 INFO:teuthology.orchestra.run.trial009.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-21T02:58:16.438 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial009 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-21T02:58:16.440 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T02:58:16.456 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T02:58:16.473 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial009 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-21T02:58:16.479 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} duration: 1597.248797416687 failure_reason: 'Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial009 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=36d388f27052dd940bdbaf543bb05bdba0acc155 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-21T02:58:16.480 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T02:58:16.601 INFO:teuthology.run:FAIL