2026-02-22T23:18:11.059 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-22T23:18:11.073 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T23:18:11.152 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64382 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '64382' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5907 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDXrfm7tk6LK7Uzg+OvhG+RmbkWysEkATpyFSORPOyhwTWp7GgIxf1S7AzL+HXcTo+PB/FP+3MbhrP/0xFaVlXE= tasks: - install: null - workunit: basedir: qa/standalone clients: all: - osd teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-22_19:35:40 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-22T23:18:11.152 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-22T23:18:11.153 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-22T23:18:11.153 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-22T23:18:11.155 INFO:teuthology.task.internal:Checking packages... 2026-02-22T23:18:11.155 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-22T23:18:11.155 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-22T23:18:11.155 INFO:teuthology.packaging:ref: None 2026-02-22T23:18:11.155 INFO:teuthology.packaging:tag: None 2026-02-22T23:18:11.155 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:18:11.155 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:18:11.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:18:11.283 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:18:11.285 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-22T23:18:11.296 INFO:teuthology.task.internal:no buildpackages task found 2026-02-22T23:18:11.296 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-22T23:18:11.309 INFO:teuthology.task.internal:Saving configuration 2026-02-22T23:18:11.313 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-22T23:18:11.324 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-22T23:18:11.385 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial122.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64382', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 23:14:41.878260', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFMZovhuaKqRdZw9M4GGJOqk1av+I8NfUKwef3yoyVzl'} 2026-02-22T23:18:11.385 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-22T23:18:11.399 INFO:teuthology.task.internal:roles: ubuntu@trial122.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-22T23:18:11.400 INFO:teuthology.run_tasks:Running task console_log... 2026-02-22T23:18:11.475 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7b0cba5eac00>, signals=[15]) 2026-02-22T23:18:11.475 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-22T23:18:11.486 INFO:teuthology.task.internal:Opening connections... 2026-02-22T23:18:11.486 DEBUG:teuthology.task.internal:connecting to ubuntu@trial122.front.sepia.ceph.com 2026-02-22T23:18:11.487 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:18:11.551 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-22T23:18:11.562 DEBUG:teuthology.orchestra.run.trial122:> uname -m 2026-02-22T23:18:11.629 INFO:teuthology.orchestra.run.trial122.stdout:x86_64 2026-02-22T23:18:11.629 DEBUG:teuthology.orchestra.run.trial122:> cat /etc/os-release 2026-02-22T23:18:11.676 INFO:teuthology.orchestra.run.trial122.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-22T23:18:11.676 INFO:teuthology.orchestra.run.trial122.stdout:NAME="Ubuntu" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:VERSION_ID="22.04" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:VERSION_CODENAME=jammy 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:ID=ubuntu 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:ID_LIKE=debian 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-22T23:18:11.677 INFO:teuthology.orchestra.run.trial122.stdout:UBUNTU_CODENAME=jammy 2026-02-22T23:18:11.678 INFO:teuthology.lock.ops:Updating trial122.front.sepia.ceph.com on lock server 2026-02-22T23:18:11.770 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-22T23:18:11.782 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-22T23:18:11.794 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-22T23:18:11.794 DEBUG:teuthology.orchestra.run.trial122:> test '!' -e /home/ubuntu/cephtest 2026-02-22T23:18:11.797 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-22T23:18:11.807 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-22T23:18:11.807 DEBUG:teuthology.orchestra.run.trial122:> test -z $(ls -A /var/lib/ceph) 2026-02-22T23:18:11.845 INFO:teuthology.orchestra.run.trial122.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-22T23:18:11.846 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-22T23:18:11.918 INFO:teuthology.run_tasks:Running task kernel... 2026-02-22T23:18:11.937 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-22T23:18:11.937 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-22T23:18:11.937 DEBUG:teuthology.orchestra.run.trial122:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T23:18:11.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:18:11.940 DEBUG:teuthology.orchestra.run.trial122:> uname -r 2026-02-22T23:18:11.984 INFO:teuthology.orchestra.run.trial122.stdout:5.15.0-170-generic 2026-02-22T23:18:11.984 INFO:teuthology.task.kernel:Running kernel on trial122: 5.15.0-170-generic 2026-02-22T23:18:11.984 DEBUG:teuthology.orchestra.run.trial122:> sudo apt-get clean 2026-02-22T23:18:12.042 DEBUG:teuthology.orchestra.run.trial122:> sudo apt-get update 2026-02-22T23:18:12.242 INFO:teuthology.orchestra.run.trial122.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-22T23:18:12.249 INFO:teuthology.orchestra.run.trial122.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-22T23:18:12.288 INFO:teuthology.orchestra.run.trial122.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-22T23:18:12.303 INFO:teuthology.orchestra.run.trial122.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-22T23:18:12.383 INFO:teuthology.orchestra.run.trial122.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-22T23:18:12.450 INFO:teuthology.orchestra.run.trial122.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-22T23:18:12.454 INFO:teuthology.orchestra.run.trial122.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-22T23:18:12.454 INFO:teuthology.orchestra.run.trial122.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-22T23:18:12.462 INFO:teuthology.orchestra.run.trial122.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-22T23:18:12.464 INFO:teuthology.orchestra.run.trial122.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-22T23:18:12.464 INFO:teuthology.orchestra.run.trial122.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-22T23:18:12.490 INFO:teuthology.orchestra.run.trial122.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-22T23:18:12.493 INFO:teuthology.orchestra.run.trial122.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-22T23:18:12.503 INFO:teuthology.orchestra.run.trial122.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-22T23:18:12.505 INFO:teuthology.orchestra.run.trial122.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-22T23:18:12.505 INFO:teuthology.orchestra.run.trial122.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-22T23:18:12.509 INFO:teuthology.orchestra.run.trial122.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-22T23:18:12.509 INFO:teuthology.orchestra.run.trial122.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-22T23:18:12.526 INFO:teuthology.orchestra.run.trial122.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-22T23:18:16.350 INFO:teuthology.orchestra.run.trial122.stdout:Fetched 22.6 MB in 2s (13.6 MB/s) 2026-02-22T23:18:16.868 INFO:teuthology.orchestra.run.trial122.stdout:Reading package lists... 2026-02-22T23:18:16.881 DEBUG:teuthology.orchestra.run.trial122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-22T23:18:16.914 INFO:teuthology.orchestra.run.trial122.stdout:Reading package lists... 2026-02-22T23:18:17.020 INFO:teuthology.orchestra.run.trial122.stdout:Building dependency tree... 2026-02-22T23:18:17.021 INFO:teuthology.orchestra.run.trial122.stdout:Reading state information... 2026-02-22T23:18:17.106 INFO:teuthology.orchestra.run.trial122.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-22T23:18:17.106 INFO:teuthology.orchestra.run.trial122.stdout:linux-image-generic set to manually installed. 2026-02-22T23:18:17.106 INFO:teuthology.orchestra.run.trial122.stdout:The following packages were automatically installed and are no longer required: 2026-02-22T23:18:17.106 INFO:teuthology.orchestra.run.trial122.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-22T23:18:17.107 INFO:teuthology.orchestra.run.trial122.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-22T23:18:17.107 INFO:teuthology.orchestra.run.trial122.stdout: sg3-utils-udev 2026-02-22T23:18:17.107 INFO:teuthology.orchestra.run.trial122.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-22T23:18:17.125 INFO:teuthology.orchestra.run.trial122.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-22T23:18:17.127 DEBUG:teuthology.orchestra.run.trial122:> dpkg -s linux-image-generic 2026-02-22T23:18:17.180 INFO:teuthology.orchestra.run.trial122.stdout:Package: linux-image-generic 2026-02-22T23:18:17.180 INFO:teuthology.orchestra.run.trial122.stdout:Status: install ok installed 2026-02-22T23:18:17.180 INFO:teuthology.orchestra.run.trial122.stdout:Priority: optional 2026-02-22T23:18:17.180 INFO:teuthology.orchestra.run.trial122.stdout:Section: kernel 2026-02-22T23:18:17.180 INFO:teuthology.orchestra.run.trial122.stdout:Installed-Size: 21 2026-02-22T23:18:17.180 INFO:teuthology.orchestra.run.trial122.stdout:Maintainer: Ubuntu Kernel Team 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Architecture: amd64 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Source: linux-meta 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Version: 5.15.0.170.159 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Recommends: thermald 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout:Description: Generic Linux kernel image 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout: This package will always depend on the latest generic kernel image 2026-02-22T23:18:17.181 INFO:teuthology.orchestra.run.trial122.stdout: available. 2026-02-22T23:18:17.182 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T23:18:17.182 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-22T23:18:17.182 DEBUG:teuthology.orchestra.run.trial122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T23:18:17.228 INFO:teuthology.orchestra.run.trial122.stdout:ttyS1 2026-02-22T23:18:17.263 DEBUG:teuthology.parallel:result is None 2026-02-22T23:18:17.263 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-22T23:18:17.275 INFO:teuthology.task.internal:Creating test directory... 2026-02-22T23:18:17.276 DEBUG:teuthology.orchestra.run.trial122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T23:18:17.308 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-22T23:18:17.321 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-22T23:18:17.337 INFO:teuthology.task.internal:Creating archive directory... 2026-02-22T23:18:17.338 DEBUG:teuthology.orchestra.run.trial122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T23:18:17.357 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-22T23:18:17.369 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-22T23:18:17.369 DEBUG:teuthology.orchestra.run.trial122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T23:18:17.405 INFO:teuthology.orchestra.run.trial122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:18:17.412 INFO:teuthology.orchestra.run.trial122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:18:17.413 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-22T23:18:17.425 INFO:teuthology.task.internal:Configuring sudo... 2026-02-22T23:18:17.426 DEBUG:teuthology.orchestra.run.trial122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T23:18:17.465 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-22T23:18:17.485 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-22T23:18:17.485 DEBUG:teuthology.orchestra.run.trial122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T23:18:17.513 DEBUG:teuthology.orchestra.run.trial122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:18:17.557 DEBUG:teuthology.orchestra.run.trial122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:18:17.601 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:18:17.601 DEBUG:teuthology.orchestra.run.trial122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T23:18:17.652 DEBUG:teuthology.orchestra.run.trial122:> sudo service rsyslog restart 2026-02-22T23:18:17.710 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-22T23:18:17.723 INFO:teuthology.task.internal:Starting timer... 2026-02-22T23:18:17.723 INFO:teuthology.run_tasks:Running task pcp... 2026-02-22T23:18:17.745 INFO:teuthology.run_tasks:Running task selinux... 2026-02-22T23:18:17.766 DEBUG:teuthology.task.selinux:Excluding trial122: OS 'ubuntu' does not support SELinux 2026-02-22T23:18:17.767 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-22T23:18:17.767 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-22T23:18:17.767 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-22T23:18:17.767 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-22T23:18:17.783 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-22T23:18:17.796 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-22T23:18:17.797 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-22T23:20:40.796 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial122.front.sepia.ceph.com')] 2026-02-22T23:20:40.797 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial122.front.sepia.ceph.com' 2026-02-22T23:20:40.798 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T23:20:40.890 DEBUG:teuthology.orchestra.run.trial122:> true 2026-02-22T23:20:41.139 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial122.front.sepia.ceph.com' 2026-02-22T23:20:41.139 INFO:teuthology.run_tasks:Running task clock... 2026-02-22T23:20:41.155 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-22T23:20:41.156 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T23:20:41.156 DEBUG:teuthology.orchestra.run.trial122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:20:41.198 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-22T23:20:41.198 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Command line: ntpd -gq 2026-02-22T23:20:41.198 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: ---------------------------------------------------- 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: ntp-4 is maintained by Network Time Foundation, 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: corporation. Support and training for ntp-4 are 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: available at https://www.nwtime.org/support 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: ---------------------------------------------------- 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: proto: precision = 0.020 usec (-25) 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: basedate set to 2022-02-04 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: gps base set to 2022-02-06 (week 2196) 2026-02-22T23:20:41.199 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-22T23:20:41.200 INFO:teuthology.orchestra.run.trial122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-22T23:20:41.200 INFO:teuthology.orchestra.run.trial122.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: restrict ::: KOD does nothing without LIMITED. 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listen and drop on 0 v6wildcard [::]:123 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listen normally on 2 lo 127.0.0.1:123 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listen normally on 3 enp1s0f0 10.20.193.122:123 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listen normally on 4 lo [::1]:123 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:61b0%2]:123 2026-02-22T23:20:41.201 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:41 ntpd[9424]: Listening on routing socket on fd #22 for interface updates 2026-02-22T23:20:42.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:42 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:42.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:42 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:42.226 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:42 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:42.227 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:42 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:42.227 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:42 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:42.227 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:42 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:43.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:43 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:43.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:43 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:43.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:43 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:43.237 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:43 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:43.238 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:43 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:43.238 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:43 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:44.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:44.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:44.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:44.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:44.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:44.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:44.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:44.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:44.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:44.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:44.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:44.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:44 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:45.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:45 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:45.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:45 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:45.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:45 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:45.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:45 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:45.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:45 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:45.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:45 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:46.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:46.199 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:46.199 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:46.199 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:46.199 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:46.199 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:46.211 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:46.211 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:46.212 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:46.212 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:46.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:46.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:46.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:46.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:46.280 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:46.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:46.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:46.281 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:46 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:47.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:47 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:47.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:47 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:47.235 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:47 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:47.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:47 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:47.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:47 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:47.236 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:47 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:48.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:48.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:48.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:48.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:48.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:48.198 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:48.211 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:48.211 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:48.211 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:48.212 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:48.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T23:20:48.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T23:20:48.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T23:20:48.225 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T23:20:48.226 INFO:teuthology.orchestra.run.trial122.stdout:22 Feb 23:20:48 ntpd[9424]: ntpd: time set +0.158855 s 2026-02-22T23:20:48.226 INFO:teuthology.orchestra.run.trial122.stdout:ntpd: time set +0.158855s 2026-02-22T23:20:48.227 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-22T23:20:48.227 INFO:teuthology.orchestra.run.trial122.stderr:22 Feb 23:20:48 ntpd[9424]: can't open /var/log/ntpstats/loopstats.20260222: Permission denied 2026-02-22T23:20:48.248 INFO:teuthology.orchestra.run.trial122.stdout:No association ID's returned 2026-02-22T23:20:48.249 INFO:teuthology.run_tasks:Running task install... 2026-02-22T23:20:48.277 DEBUG:teuthology.task.install:project ceph 2026-02-22T23:20:48.277 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}} 2026-02-22T23:20:48.277 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'} 2026-02-22T23:20:48.278 INFO:teuthology.task.install:Using flavor: default 2026-02-22T23:20:48.297 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-22T23:20:48.297 INFO:teuthology.task.install:extra packages: [] 2026-02-22T23:20:48.298 DEBUG:teuthology.orchestra.run.trial122:> sudo apt-key list | grep Ceph 2026-02-22T23:20:48.351 INFO:teuthology.orchestra.run.trial122.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-22T23:20:48.381 INFO:teuthology.orchestra.run.trial122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-22T23:20:48.381 INFO:teuthology.orchestra.run.trial122.stdout:uid [ unknown] Ceph.com (release key) 2026-02-22T23:20:48.383 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-22T23:20:48.384 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:20:48.515 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default/ 2026-02-22T23:20:48.515 INFO:teuthology.task.install.deb:Package version is 19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:20:48.644 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:20:48.644 DEBUG:teuthology.orchestra.run.trial122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-22T23:20:48.654 DEBUG:teuthology.orchestra.run.trial122:> sudo apt-get update 2026-02-22T23:20:48.829 INFO:teuthology.orchestra.run.trial122.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy InRelease 2026-02-22T23:20:48.840 INFO:teuthology.orchestra.run.trial122.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy Release [7662 B] 2026-02-22T23:20:48.852 INFO:teuthology.orchestra.run.trial122.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-22T23:20:48.863 INFO:teuthology.orchestra.run.trial122.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-22T23:20:49.100 INFO:teuthology.orchestra.run.trial122.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-22T23:20:49.166 INFO:teuthology.orchestra.run.trial122.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-22T23:20:49.238 INFO:teuthology.orchestra.run.trial122.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-22T23:20:49.310 INFO:teuthology.orchestra.run.trial122.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-22T23:20:49.455 INFO:teuthology.orchestra.run.trial122.stdout:Fetched 25.8 kB in 1s (40.2 kB/s) 2026-02-22T23:20:49.966 INFO:teuthology.orchestra.run.trial122.stdout:Reading package lists... 2026-02-22T23:20:49.979 DEBUG:teuthology.orchestra.run.trial122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-689-gf90f4eae-1jammy cephadm=19.2.3-689-gf90f4eae-1jammy ceph-mds=19.2.3-689-gf90f4eae-1jammy ceph-mgr=19.2.3-689-gf90f4eae-1jammy ceph-common=19.2.3-689-gf90f4eae-1jammy ceph-fuse=19.2.3-689-gf90f4eae-1jammy ceph-test=19.2.3-689-gf90f4eae-1jammy ceph-volume=19.2.3-689-gf90f4eae-1jammy radosgw=19.2.3-689-gf90f4eae-1jammy python3-rados=19.2.3-689-gf90f4eae-1jammy python3-rgw=19.2.3-689-gf90f4eae-1jammy python3-cephfs=19.2.3-689-gf90f4eae-1jammy python3-rbd=19.2.3-689-gf90f4eae-1jammy libcephfs2=19.2.3-689-gf90f4eae-1jammy libcephfs-dev=19.2.3-689-gf90f4eae-1jammy librados2=19.2.3-689-gf90f4eae-1jammy librbd1=19.2.3-689-gf90f4eae-1jammy rbd-fuse=19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:20:50.013 INFO:teuthology.orchestra.run.trial122.stdout:Reading package lists... 2026-02-22T23:20:50.116 INFO:teuthology.orchestra.run.trial122.stdout:Building dependency tree... 2026-02-22T23:20:50.117 INFO:teuthology.orchestra.run.trial122.stdout:Reading state information... 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout:The following packages were automatically installed and are no longer required: 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout: sg3-utils-udev 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout:The following additional packages will be installed: 2026-02-22T23:20:50.205 INFO:teuthology.orchestra.run.trial122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-22T23:20:50.206 INFO:teuthology.orchestra.run.trial122.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout:Suggested packages: 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout:Recommended packages: 2026-02-22T23:20:50.207 INFO:teuthology.orchestra.run.trial122.stdout: btrfs-tools 2026-02-22T23:20:50.240 INFO:teuthology.orchestra.run.trial122.stdout:The following NEW packages will be installed: 2026-02-22T23:20:50.240 INFO:teuthology.orchestra.run.trial122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-22T23:20:50.240 INFO:teuthology.orchestra.run.trial122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-22T23:20:50.240 INFO:teuthology.orchestra.run.trial122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-22T23:20:50.241 INFO:teuthology.orchestra.run.trial122.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-22T23:20:50.250 INFO:teuthology.orchestra.run.trial122.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-22T23:20:50.251 INFO:teuthology.orchestra.run.trial122.stdout: xmlstarlet zip 2026-02-22T23:20:50.340 INFO:teuthology.orchestra.run.trial122.stdout:0 upgraded, 108 newly installed, 0 to remove and 13 not upgraded. 2026-02-22T23:20:50.340 INFO:teuthology.orchestra.run.trial122.stdout:Need to get 178 MB of archives. 2026-02-22T23:20:50.340 INFO:teuthology.orchestra.run.trial122.stdout:After this operation, 809 MB of additional disk space will be used. 2026-02-22T23:20:50.340 INFO:teuthology.orchestra.run.trial122.stdout:Get:1 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-22T23:20:50.384 INFO:teuthology.orchestra.run.trial122.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-22T23:20:50.385 INFO:teuthology.orchestra.run.trial122.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-22T23:20:50.394 INFO:teuthology.orchestra.run.trial122.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-22T23:20:50.423 INFO:teuthology.orchestra.run.trial122.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-22T23:20:50.424 INFO:teuthology.orchestra.run.trial122.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-22T23:20:50.429 INFO:teuthology.orchestra.run.trial122.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-22T23:20:50.431 INFO:teuthology.orchestra.run.trial122.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-22T23:20:50.432 INFO:teuthology.orchestra.run.trial122.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-22T23:20:50.432 INFO:teuthology.orchestra.run.trial122.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-22T23:20:50.433 INFO:teuthology.orchestra.run.trial122.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-22T23:20:50.437 INFO:teuthology.orchestra.run.trial122.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-22T23:20:50.438 INFO:teuthology.orchestra.run.trial122.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-22T23:20:50.440 INFO:teuthology.orchestra.run.trial122.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-22T23:20:50.443 INFO:teuthology.orchestra.run.trial122.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-22T23:20:50.443 INFO:teuthology.orchestra.run.trial122.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-22T23:20:50.445 INFO:teuthology.orchestra.run.trial122.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-22T23:20:50.446 INFO:teuthology.orchestra.run.trial122.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-22T23:20:50.453 INFO:teuthology.orchestra.run.trial122.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-22T23:20:50.453 INFO:teuthology.orchestra.run.trial122.stdout:Get:20 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-22T23:20:50.454 INFO:teuthology.orchestra.run.trial122.stdout:Get:21 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-22T23:20:50.454 INFO:teuthology.orchestra.run.trial122.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-22T23:20:50.454 INFO:teuthology.orchestra.run.trial122.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-22T23:20:50.455 INFO:teuthology.orchestra.run.trial122.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-22T23:20:50.462 INFO:teuthology.orchestra.run.trial122.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-22T23:20:50.462 INFO:teuthology.orchestra.run.trial122.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-22T23:20:50.463 INFO:teuthology.orchestra.run.trial122.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-22T23:20:50.463 INFO:teuthology.orchestra.run.trial122.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-22T23:20:50.471 INFO:teuthology.orchestra.run.trial122.stdout:Get:29 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-22T23:20:50.472 INFO:teuthology.orchestra.run.trial122.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-22T23:20:50.472 INFO:teuthology.orchestra.run.trial122.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-22T23:20:50.473 INFO:teuthology.orchestra.run.trial122.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-22T23:20:50.473 INFO:teuthology.orchestra.run.trial122.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-22T23:20:50.473 INFO:teuthology.orchestra.run.trial122.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-22T23:20:50.481 INFO:teuthology.orchestra.run.trial122.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-22T23:20:50.481 INFO:teuthology.orchestra.run.trial122.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-22T23:20:50.482 INFO:teuthology.orchestra.run.trial122.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-22T23:20:50.485 INFO:teuthology.orchestra.run.trial122.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-22T23:20:50.490 INFO:teuthology.orchestra.run.trial122.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-22T23:20:50.491 INFO:teuthology.orchestra.run.trial122.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-22T23:20:50.491 INFO:teuthology.orchestra.run.trial122.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-22T23:20:50.492 INFO:teuthology.orchestra.run.trial122.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-22T23:20:50.493 INFO:teuthology.orchestra.run.trial122.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-22T23:20:50.494 INFO:teuthology.orchestra.run.trial122.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-22T23:20:50.500 INFO:teuthology.orchestra.run.trial122.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-22T23:20:50.500 INFO:teuthology.orchestra.run.trial122.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-22T23:20:50.501 INFO:teuthology.orchestra.run.trial122.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-22T23:20:50.513 INFO:teuthology.orchestra.run.trial122.stdout:Get:48 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-689-gf90f4eae-1jammy [3597 kB] 2026-02-22T23:20:50.515 INFO:teuthology.orchestra.run.trial122.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-22T23:20:50.516 INFO:teuthology.orchestra.run.trial122.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-22T23:20:50.517 INFO:teuthology.orchestra.run.trial122.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-22T23:20:50.526 INFO:teuthology.orchestra.run.trial122.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-22T23:20:50.526 INFO:teuthology.orchestra.run.trial122.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-22T23:20:50.526 INFO:teuthology.orchestra.run.trial122.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-22T23:20:50.527 INFO:teuthology.orchestra.run.trial122.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-22T23:20:50.527 INFO:teuthology.orchestra.run.trial122.stdout:Get:56 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-22T23:20:50.527 INFO:teuthology.orchestra.run.trial122.stdout:Get:57 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-22T23:20:50.528 INFO:teuthology.orchestra.run.trial122.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-22T23:20:50.529 INFO:teuthology.orchestra.run.trial122.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-22T23:20:50.529 INFO:teuthology.orchestra.run.trial122.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-22T23:20:50.537 INFO:teuthology.orchestra.run.trial122.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-22T23:20:50.539 INFO:teuthology.orchestra.run.trial122.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-22T23:20:50.540 INFO:teuthology.orchestra.run.trial122.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-22T23:20:50.540 INFO:teuthology.orchestra.run.trial122.stdout:Get:64 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-22T23:20:50.541 INFO:teuthology.orchestra.run.trial122.stdout:Get:65 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-22T23:20:50.543 INFO:teuthology.orchestra.run.trial122.stdout:Get:66 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-22T23:20:50.547 INFO:teuthology.orchestra.run.trial122.stdout:Get:67 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-22T23:20:50.548 INFO:teuthology.orchestra.run.trial122.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-22T23:20:50.549 INFO:teuthology.orchestra.run.trial122.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-22T23:20:50.549 INFO:teuthology.orchestra.run.trial122.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-22T23:20:50.556 INFO:teuthology.orchestra.run.trial122.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-22T23:20:50.557 INFO:teuthology.orchestra.run.trial122.stdout:Get:72 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-22T23:20:50.558 INFO:teuthology.orchestra.run.trial122.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-22T23:20:50.562 INFO:teuthology.orchestra.run.trial122.stdout:Get:74 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-22T23:20:50.563 INFO:teuthology.orchestra.run.trial122.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-22T23:20:50.563 INFO:teuthology.orchestra.run.trial122.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-22T23:20:50.566 INFO:teuthology.orchestra.run.trial122.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-22T23:20:50.566 INFO:teuthology.orchestra.run.trial122.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-22T23:20:50.975 INFO:teuthology.orchestra.run.trial122.stdout:Get:79 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-689-gf90f4eae-1jammy [3256 kB] 2026-02-22T23:20:51.128 INFO:teuthology.orchestra.run.trial122.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-689-gf90f4eae-1jammy [979 kB] 2026-02-22T23:20:51.192 INFO:teuthology.orchestra.run.trial122.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-689-gf90f4eae-1jammy [357 kB] 2026-02-22T23:20:51.211 INFO:teuthology.orchestra.run.trial122.stdout:Get:82 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-689-gf90f4eae-1jammy [32.9 kB] 2026-02-22T23:20:51.212 INFO:teuthology.orchestra.run.trial122.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-689-gf90f4eae-1jammy [184 kB] 2026-02-22T23:20:51.214 INFO:teuthology.orchestra.run.trial122.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-689-gf90f4eae-1jammy [70.0 kB] 2026-02-22T23:20:51.216 INFO:teuthology.orchestra.run.trial122.stdout:Get:85 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-689-gf90f4eae-1jammy [335 kB] 2026-02-22T23:20:51.221 INFO:teuthology.orchestra.run.trial122.stdout:Get:86 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-689-gf90f4eae-1jammy [6935 kB] 2026-02-22T23:20:51.497 INFO:teuthology.orchestra.run.trial122.stdout:Get:87 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-689-gf90f4eae-1jammy [112 kB] 2026-02-22T23:20:51.513 INFO:teuthology.orchestra.run.trial122.stdout:Get:88 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-689-gf90f4eae-1jammy [470 kB] 2026-02-22T23:20:51.522 INFO:teuthology.orchestra.run.trial122.stdout:Get:89 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-689-gf90f4eae-1jammy [26.5 MB] 2026-02-22T23:20:52.479 INFO:teuthology.orchestra.run.trial122.stdout:Get:90 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-689-gf90f4eae-1jammy [5178 kB] 2026-02-22T23:20:52.675 INFO:teuthology.orchestra.run.trial122.stdout:Get:91 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-689-gf90f4eae-1jammy [248 kB] 2026-02-22T23:20:52.678 INFO:teuthology.orchestra.run.trial122.stdout:Get:92 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-689-gf90f4eae-1jammy [125 kB] 2026-02-22T23:20:52.680 INFO:teuthology.orchestra.run.trial122.stdout:Get:93 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-689-gf90f4eae-1jammy [1081 kB] 2026-02-22T23:20:52.727 INFO:teuthology.orchestra.run.trial122.stdout:Get:94 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-689-gf90f4eae-1jammy [6239 kB] 2026-02-22T23:20:52.947 INFO:teuthology.orchestra.run.trial122.stdout:Get:95 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-689-gf90f4eae-1jammy [23.0 MB] 2026-02-22T23:20:53.541 INFO:teuthology.orchestra.run.trial122.stdout:Get:96 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-689-gf90f4eae-1jammy [14.2 kB] 2026-02-22T23:20:53.541 INFO:teuthology.orchestra.run.trial122.stdout:Get:97 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-689-gf90f4eae-1jammy [1173 kB] 2026-02-22T23:20:53.557 INFO:teuthology.orchestra.run.trial122.stdout:Get:98 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-689-gf90f4eae-1jammy [2503 kB] 2026-02-22T23:20:53.598 INFO:teuthology.orchestra.run.trial122.stdout:Get:99 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-689-gf90f4eae-1jammy [798 kB] 2026-02-22T23:20:53.610 INFO:teuthology.orchestra.run.trial122.stdout:Get:100 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-689-gf90f4eae-1jammy [157 kB] 2026-02-22T23:20:53.613 INFO:teuthology.orchestra.run.trial122.stdout:Get:101 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-689-gf90f4eae-1jammy [2396 kB] 2026-02-22T23:20:53.653 INFO:teuthology.orchestra.run.trial122.stdout:Get:102 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-689-gf90f4eae-1jammy [8625 kB] 2026-02-22T23:20:53.795 INFO:teuthology.orchestra.run.trial122.stdout:Get:103 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-689-gf90f4eae-1jammy [14.3 kB] 2026-02-22T23:20:53.795 INFO:teuthology.orchestra.run.trial122.stdout:Get:104 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-689-gf90f4eae-1jammy [52.1 MB] 2026-02-22T23:20:55.254 INFO:teuthology.orchestra.run.trial122.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-689-gf90f4eae-1jammy [135 kB] 2026-02-22T23:20:55.255 INFO:teuthology.orchestra.run.trial122.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-689-gf90f4eae-1jammy [41.0 kB] 2026-02-22T23:20:55.255 INFO:teuthology.orchestra.run.trial122.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-689-gf90f4eae-1jammy [13.7 MB] 2026-02-22T23:20:55.731 INFO:teuthology.orchestra.run.trial122.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-689-gf90f4eae-1jammy [92.3 kB] 2026-02-22T23:20:56.053 INFO:teuthology.orchestra.run.trial122.stdout:Fetched 178 MB in 5s (32.4 MB/s) 2026-02-22T23:20:56.070 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-22T23:20:56.225 INFO:teuthology.orchestra.run.trial122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-22T23:20:56.227 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-22T23:20:56.229 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-22T23:20:56.250 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-22T23:20:56.262 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-22T23:20:56.263 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-22T23:20:56.278 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-22T23:20:56.290 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-22T23:20:56.291 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-22T23:20:56.312 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-22T23:20:56.320 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T23:20:56.325 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:56.356 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-22T23:20:56.364 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T23:20:56.364 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:56.381 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-22T23:20:56.389 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T23:20:56.389 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:20:56.410 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-22T23:20:56.417 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-22T23:20:56.418 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-22T23:20:56.434 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package librados2. 2026-02-22T23:20:56.442 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../007-librados2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.442 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking librados2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.482 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libnbd0. 2026-02-22T23:20:56.490 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-22T23:20:56.490 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-22T23:20:56.505 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package librbd1. 2026-02-22T23:20:56.513 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../009-librbd1_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.514 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking librbd1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.554 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libcephfs2. 2026-02-22T23:20:56.562 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.563 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libcephfs2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.582 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-rados. 2026-02-22T23:20:56.590 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../011-python3-rados_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.591 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-rados (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.608 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-22T23:20:56.617 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:56.617 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-ceph-argparse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.631 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-cephfs. 2026-02-22T23:20:56.639 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.640 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-cephfs (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.654 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-22T23:20:56.662 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:56.663 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.679 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-22T23:20:56.687 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-22T23:20:56.687 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-22T23:20:56.702 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-prettytable. 2026-02-22T23:20:56.711 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-22T23:20:56.712 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-22T23:20:56.725 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-rbd. 2026-02-22T23:20:56.733 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.733 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-rbd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:56.750 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-22T23:20:56.758 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-22T23:20:56.759 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-22T23:20:56.777 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-22T23:20:56.785 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-22T23:20:56.786 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-22T23:20:56.802 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-22T23:20:56.810 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-22T23:20:56.811 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-22T23:20:56.829 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package lua5.1. 2026-02-22T23:20:56.837 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-22T23:20:56.837 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-22T23:20:56.856 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package lua-any. 2026-02-22T23:20:56.865 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-22T23:20:56.865 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-22T23:20:56.878 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package zip. 2026-02-22T23:20:56.886 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-22T23:20:56.887 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking zip (3.0-12build2) ... 2026-02-22T23:20:56.903 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package unzip. 2026-02-22T23:20:56.911 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-22T23:20:56.911 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-22T23:20:56.930 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package luarocks. 2026-02-22T23:20:56.938 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-22T23:20:56.938 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-22T23:20:56.975 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package librgw2. 2026-02-22T23:20:56.983 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../026-librgw2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:56.984 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking librgw2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:57.054 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-rgw. 2026-02-22T23:20:57.063 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:57.063 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-rgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:57.078 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-22T23:20:57.086 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-22T23:20:57.086 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-22T23:20:57.101 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libradosstriper1. 2026-02-22T23:20:57.109 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:57.110 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libradosstriper1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:57.128 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-common. 2026-02-22T23:20:57.136 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../030-ceph-common_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:57.136 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:57.348 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-base. 2026-02-22T23:20:57.357 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../031-ceph-base_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:57.362 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-base (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:57.423 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-22T23:20:57.432 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-22T23:20:57.432 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-22T23:20:57.447 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-cheroot. 2026-02-22T23:20:57.455 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-22T23:20:57.456 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-22T23:20:57.472 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-22T23:20:57.481 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-22T23:20:57.481 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-22T23:20:57.496 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-22T23:20:57.505 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-22T23:20:57.505 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-22T23:20:57.519 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-22T23:20:57.527 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-22T23:20:57.528 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-22T23:20:57.542 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-tempora. 2026-02-22T23:20:57.550 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-22T23:20:57.550 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-22T23:20:57.564 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-portend. 2026-02-22T23:20:57.573 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-22T23:20:57.573 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-22T23:20:57.587 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-22T23:20:57.596 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-22T23:20:57.596 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-22T23:20:57.611 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-22T23:20:57.620 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-22T23:20:57.620 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-22T23:20:57.643 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-natsort. 2026-02-22T23:20:57.651 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-22T23:20:57.652 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-22T23:20:57.668 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-logutils. 2026-02-22T23:20:57.677 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-22T23:20:57.677 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-22T23:20:57.692 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-mako. 2026-02-22T23:20:57.700 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-22T23:20:57.701 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-22T23:20:57.718 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-22T23:20:57.726 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-22T23:20:57.727 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-22T23:20:57.741 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-22T23:20:57.749 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-22T23:20:57.749 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-22T23:20:57.763 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-webob. 2026-02-22T23:20:57.772 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-22T23:20:57.772 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-22T23:20:57.788 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-waitress. 2026-02-22T23:20:57.796 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-22T23:20:57.798 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-22T23:20:57.814 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-tempita. 2026-02-22T23:20:57.822 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-22T23:20:57.822 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-22T23:20:57.836 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-paste. 2026-02-22T23:20:57.844 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-22T23:20:57.845 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-22T23:20:57.872 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-22T23:20:57.880 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-22T23:20:57.881 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-22T23:20:57.895 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-22T23:20:57.903 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-22T23:20:57.903 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-22T23:20:57.918 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-webtest. 2026-02-22T23:20:57.926 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-22T23:20:57.927 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-22T23:20:57.942 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pecan. 2026-02-22T23:20:57.951 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-22T23:20:57.951 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-22T23:20:57.975 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-22T23:20:57.983 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-22T23:20:57.984 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-22T23:20:58.003 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-22T23:20:58.011 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:58.011 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mgr-modules-core (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.039 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-22T23:20:58.048 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.048 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.063 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mgr. 2026-02-22T23:20:58.071 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.071 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mgr (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.094 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mon. 2026-02-22T23:20:58.102 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.102 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mon (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.161 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-22T23:20:58.169 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-22T23:20:58.169 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-22T23:20:58.185 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-osd. 2026-02-22T23:20:58.194 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.194 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-osd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.346 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph. 2026-02-22T23:20:58.355 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../061-ceph_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.355 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.369 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-fuse. 2026-02-22T23:20:58.377 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.378 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.400 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mds. 2026-02-22T23:20:58.409 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.410 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mds (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.443 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package cephadm. 2026-02-22T23:20:58.451 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../064-cephadm_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:58.451 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.467 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-22T23:20:58.476 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-22T23:20:58.476 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-22T23:20:58.498 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-22T23:20:58.506 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:58.507 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mgr-cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.526 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-22T23:20:58.535 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-22T23:20:58.535 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-22T23:20:58.551 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-routes. 2026-02-22T23:20:58.559 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-22T23:20:58.560 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-22T23:20:58.579 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-22T23:20:58.588 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:58.588 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mgr-dashboard (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:58.794 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-22T23:20:58.803 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-22T23:20:58.803 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-22T23:20:58.845 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-joblib. 2026-02-22T23:20:58.853 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-22T23:20:58.853 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-22T23:20:58.881 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-22T23:20:58.889 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-22T23:20:58.890 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-22T23:20:58.905 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-sklearn. 2026-02-22T23:20:58.913 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-22T23:20:58.913 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-22T23:20:58.995 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-22T23:20:59.004 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:59.004 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:59.170 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-cachetools. 2026-02-22T23:20:59.178 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-22T23:20:59.179 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-22T23:20:59.194 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-rsa. 2026-02-22T23:20:59.202 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-22T23:20:59.202 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-22T23:20:59.218 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-google-auth. 2026-02-22T23:20:59.226 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-22T23:20:59.227 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-22T23:20:59.243 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-22T23:20:59.252 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-22T23:20:59.252 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-22T23:20:59.268 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-websocket. 2026-02-22T23:20:59.276 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-22T23:20:59.277 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-22T23:20:59.294 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-22T23:20:59.302 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-22T23:20:59.303 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-22T23:20:59.395 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-22T23:20:59.404 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:20:59.404 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:20:59.418 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-22T23:20:59.427 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-22T23:20:59.434 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-22T23:20:59.449 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-22T23:20:59.458 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-22T23:20:59.459 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-22T23:20:59.473 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package jq. 2026-02-22T23:20:59.481 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-22T23:20:59.482 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-22T23:20:59.496 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package socat. 2026-02-22T23:20:59.505 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-22T23:20:59.505 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-22T23:20:59.525 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package xmlstarlet. 2026-02-22T23:20:59.533 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-22T23:20:59.534 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-22T23:20:59.569 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-test. 2026-02-22T23:20:59.578 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../087-ceph-test_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:20:59.578 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-test (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:00.046 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package ceph-volume. 2026-02-22T23:21:00.054 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T23:21:00.054 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking ceph-volume (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:00.078 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-22T23:21:00.086 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:21:00.086 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking libcephfs-dev (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:00.102 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-22T23:21:00.110 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-22T23:21:00.110 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-22T23:21:00.131 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-22T23:21:00.140 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-22T23:21:00.140 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-22T23:21:00.157 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package nvme-cli. 2026-02-22T23:21:00.166 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-22T23:21:00.166 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-22T23:21:00.197 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package pkg-config. 2026-02-22T23:21:00.205 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-22T23:21:00.206 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-22T23:21:00.221 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-22T23:21:00.230 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-22T23:21:00.230 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-22T23:21:00.259 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-22T23:21:00.268 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-22T23:21:00.268 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-22T23:21:00.282 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pastescript. 2026-02-22T23:21:00.290 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-22T23:21:00.291 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-22T23:21:00.309 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pluggy. 2026-02-22T23:21:00.317 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-22T23:21:00.317 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-22T23:21:00.333 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-psutil. 2026-02-22T23:21:00.341 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-22T23:21:00.342 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-22T23:21:00.363 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-py. 2026-02-22T23:21:00.369 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-22T23:21:00.369 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-22T23:21:00.389 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pygments. 2026-02-22T23:21:00.397 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-22T23:21:00.398 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-22T23:21:00.439 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-22T23:21:00.447 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-22T23:21:00.448 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-22T23:21:00.463 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-toml. 2026-02-22T23:21:00.471 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-22T23:21:00.472 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-22T23:21:00.486 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-pytest. 2026-02-22T23:21:00.494 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-22T23:21:00.495 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-22T23:21:00.517 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package python3-simplejson. 2026-02-22T23:21:00.525 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-22T23:21:00.526 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-22T23:21:00.542 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-22T23:21:00.551 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-22T23:21:00.551 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-22T23:21:00.618 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package radosgw. 2026-02-22T23:21:00.626 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../106-radosgw_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:21:00.627 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking radosgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:00.740 INFO:teuthology.orchestra.run.trial122.stdout:Selecting previously unselected package rbd-fuse. 2026-02-22T23:21:00.749 INFO:teuthology.orchestra.run.trial122.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T23:21:00.749 INFO:teuthology.orchestra.run.trial122.stdout:Unpacking rbd-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:00.778 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-22T23:21:00.845 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-22T23:21:00.846 INFO:teuthology.orchestra.run.trial122.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-22T23:21:00.917 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:21:01.201 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-22T23:21:01.598 INFO:teuthology.orchestra.run.trial122.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-22T23:21:01.610 INFO:teuthology.orchestra.run.trial122.stdout:Setting up cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:01.655 INFO:teuthology.orchestra.run.trial122.stdout:Adding system user cephadm....done 2026-02-22T23:21:01.666 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-22T23:21:01.736 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-22T23:21:01.809 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-22T23:21:01.810 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-22T23:21:01.881 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-22T23:21:01.956 INFO:teuthology.orchestra.run.trial122.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-22T23:21:01.957 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-22T23:21:02.054 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-22T23:21:02.175 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-22T23:21:02.252 INFO:teuthology.orchestra.run.trial122.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-22T23:21:02.261 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-22T23:21:02.338 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-22T23:21:02.411 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-ceph-argparse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:02.479 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-22T23:21:02.480 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-22T23:21:02.481 INFO:teuthology.orchestra.run.trial122.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-22T23:21:02.482 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-22T23:21:02.482 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-22T23:21:02.483 INFO:teuthology.orchestra.run.trial122.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-22T23:21:02.500 INFO:teuthology.orchestra.run.trial122.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-22T23:21:02.502 INFO:teuthology.orchestra.run.trial122.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-22T23:21:02.503 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-22T23:21:02.504 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-22T23:21:02.627 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-22T23:21:02.712 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-22T23:21:02.791 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-22T23:21:02.876 INFO:teuthology.orchestra.run.trial122.stdout:Setting up zip (3.0-12build2) ... 2026-02-22T23:21:02.877 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-22T23:21:03.127 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-22T23:21:03.205 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-22T23:21:03.206 INFO:teuthology.orchestra.run.trial122.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-22T23:21:03.207 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-22T23:21:03.303 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-22T23:21:03.426 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-22T23:21:03.553 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-22T23:21:03.634 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-22T23:21:03.738 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-22T23:21:03.810 INFO:teuthology.orchestra.run.trial122.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-22T23:21:03.811 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:03.895 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-22T23:21:04.389 INFO:teuthology.orchestra.run.trial122.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-22T23:21:04.409 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:21:04.414 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-22T23:21:04.489 INFO:teuthology.orchestra.run.trial122.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-22T23:21:04.490 INFO:teuthology.orchestra.run.trial122.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-22T23:21:04.491 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-22T23:21:04.564 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-22T23:21:04.637 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:21:04.637 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-22T23:21:04.715 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-22T23:21:04.790 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-22T23:21:04.868 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-22T23:21:04.943 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-22T23:21:05.013 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-22T23:21:05.091 INFO:teuthology.orchestra.run.trial122.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-22T23:21:05.092 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-22T23:21:05.175 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-22T23:21:05.176 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-22T23:21:05.253 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-22T23:21:05.331 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-22T23:21:05.416 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-22T23:21:05.490 INFO:teuthology.orchestra.run.trial122.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-22T23:21:05.490 INFO:teuthology.orchestra.run.trial122.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-22T23:21:05.491 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-22T23:21:05.492 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-22T23:21:05.628 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-22T23:21:05.705 INFO:teuthology.orchestra.run.trial122.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-22T23:21:05.706 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-22T23:21:05.777 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T23:21:05.778 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-22T23:21:05.862 INFO:teuthology.orchestra.run.trial122.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-22T23:21:05.862 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-22T23:21:05.941 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-22T23:21:06.070 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-22T23:21:06.157 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-22T23:21:06.272 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-22T23:21:06.273 INFO:teuthology.orchestra.run.trial122.stdout:Setting up librados2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.274 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libsqlite3-mod-ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.275 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-22T23:21:06.790 INFO:teuthology.orchestra.run.trial122.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-22T23:21:06.802 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libcephfs2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.803 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libradosstriper1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.803 INFO:teuthology.orchestra.run.trial122.stdout:Setting up librbd1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.804 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mgr-modules-core (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.805 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:06.875 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-22T23:21:06.875 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-22T23:21:07.282 INFO:teuthology.orchestra.run.trial122.stdout:Setting up libcephfs-dev (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.283 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-rados (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.284 INFO:teuthology.orchestra.run.trial122.stdout:Setting up librgw2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.285 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-rbd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.286 INFO:teuthology.orchestra.run.trial122.stdout:Setting up rbd-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.286 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-rgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.287 INFO:teuthology.orchestra.run.trial122.stdout:Setting up python3-cephfs (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.288 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:07.317 INFO:teuthology.orchestra.run.trial122.stdout:Adding group ceph....done 2026-02-22T23:21:07.352 INFO:teuthology.orchestra.run.trial122.stdout:Adding system user ceph....done 2026-02-22T23:21:07.362 INFO:teuthology.orchestra.run.trial122.stdout:Setting system user ceph properties....done 2026-02-22T23:21:07.368 INFO:teuthology.orchestra.run.trial122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-22T23:21:07.455 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-22T23:21:07.733 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-22T23:21:08.129 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-test (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:08.130 INFO:teuthology.orchestra.run.trial122.stdout:Setting up radosgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:08.388 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:21:08.388 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:21:08.771 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-base (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:08.891 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-22T23:21:09.272 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mds (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:09.351 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-22T23:21:09.352 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-22T23:21:09.736 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mgr (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:09.816 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-22T23:21:09.816 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-22T23:21:10.199 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-osd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:10.272 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-22T23:21:10.273 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-22T23:21:10.658 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mgr-k8sevents (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:10.659 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:10.669 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mon (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:10.740 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-22T23:21:10.741 INFO:teuthology.orchestra.run.trial122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-22T23:21:11.139 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mgr-cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:11.151 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:11.152 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-mgr-dashboard (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:11.162 INFO:teuthology.orchestra.run.trial122.stdout:Setting up ceph-volume (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T23:21:11.270 INFO:teuthology.orchestra.run.trial122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-22T23:21:11.277 INFO:teuthology.orchestra.run.trial122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-22T23:21:11.521 INFO:teuthology.orchestra.run.trial122.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-22T23:21:11.605 INFO:teuthology.orchestra.run.trial122.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-22T23:21:12.108 INFO:teuthology.orchestra.run.trial122.stdout: 2026-02-22T23:21:12.109 INFO:teuthology.orchestra.run.trial122.stdout:Running kernel seems to be up-to-date. 2026-02-22T23:21:12.109 INFO:teuthology.orchestra.run.trial122.stdout: 2026-02-22T23:21:12.109 INFO:teuthology.orchestra.run.trial122.stdout:The processor microcode seems to be up-to-date. 2026-02-22T23:21:12.109 INFO:teuthology.orchestra.run.trial122.stdout: 2026-02-22T23:21:12.110 INFO:teuthology.orchestra.run.trial122.stdout:No services need to be restarted. 2026-02-22T23:21:12.110 INFO:teuthology.orchestra.run.trial122.stdout: 2026-02-22T23:21:12.110 INFO:teuthology.orchestra.run.trial122.stdout:No containers need to be restarted. 2026-02-22T23:21:12.110 INFO:teuthology.orchestra.run.trial122.stdout: 2026-02-22T23:21:12.110 INFO:teuthology.orchestra.run.trial122.stdout:No user sessions are running outdated binaries. 2026-02-22T23:21:12.111 INFO:teuthology.orchestra.run.trial122.stdout: 2026-02-22T23:21:12.111 INFO:teuthology.orchestra.run.trial122.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-22T23:21:12.905 INFO:teuthology.orchestra.run.trial122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-22T23:21:12.910 DEBUG:teuthology.parallel:result is None 2026-02-22T23:21:12.910 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:21:13.061 DEBUG:teuthology.orchestra.run.trial122:> dpkg-query -W -f '${Version}' ceph 2026-02-22T23:21:13.072 INFO:teuthology.orchestra.run.trial122.stdout:19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:21:13.073 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689-gf90f4eae-1jammy 2026-02-22T23:21:13.073 INFO:teuthology.task.install:The correct ceph version 19.2.3-689-gf90f4eae-1jammy is installed. 2026-02-22T23:21:13.075 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-22T23:21:13.075 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:21:13.075 DEBUG:teuthology.orchestra.run.trial122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:21:13.128 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-22T23:21:13.129 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:21:13.129 DEBUG:teuthology.orchestra.run.trial122:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:21:13.181 DEBUG:teuthology.orchestra.run.trial122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:21:13.232 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-22T23:21:13.232 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:21:13.232 DEBUG:teuthology.orchestra.run.trial122:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:21:13.285 DEBUG:teuthology.orchestra.run.trial122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:21:13.335 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-22T23:21:13.336 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:21:13.336 DEBUG:teuthology.orchestra.run.trial122:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:21:13.384 DEBUG:teuthology.orchestra.run.trial122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:21:13.435 INFO:teuthology.run_tasks:Running task workunit... 2026-02-22T23:21:13.456 INFO:tasks.workunit:Pulling workunits from ref f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:21:13.456 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-22T23:21:13.456 INFO:tasks.workunit:timeout=3h 2026-02-22T23:21:13.456 INFO:tasks.workunit:cleanup=True 2026-02-22T23:21:13.457 DEBUG:teuthology.orchestra.run.trial122:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-22T23:21:13.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:21:13.479 INFO:teuthology.orchestra.run.trial122.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-22T23:21:13.480 DEBUG:teuthology.orchestra.run.trial122:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-22T23:21:13.527 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-22T23:21:13.527 DEBUG:teuthology.orchestra.run.trial122:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-22T23:21:13.571 DEBUG:teuthology.orchestra.run.trial122:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:21:13.622 INFO:tasks.workunit.client.0.trial122.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-22T23:21:35.574 INFO:tasks.workunit.client.0.trial122.stderr:Note: switching to 'f90f4eaebb28b619cf76e80cbbe997656da69579'. 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr:state without impacting any branches by switching back to a branch. 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr: git switch -c 2026-02-22T23:21:35.575 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr:Or undo this operation with: 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr: git switch - 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr: 2026-02-22T23:21:35.576 INFO:tasks.workunit.client.0.trial122.stderr:HEAD is now at f90f4eaebb2 Merge branch 'wip-74712-squid' of https://github.com/idryomov/ceph into wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:21:35.584 DEBUG:teuthology.orchestra.run.trial122:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-22T23:21:35.633 DEBUG:teuthology.orchestra.run.trial122:> set -ex 2026-02-22T23:21:35.633 DEBUG:teuthology.orchestra.run.trial122:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-22T23:21:35.680 INFO:tasks.workunit:Running workunits matching osd on client.0... 2026-02-22T23:21:35.681 INFO:tasks.workunit:Running workunit osd/bad-inc-map.sh... 2026-02-22T23:21:35.681 DEBUG:teuthology.orchestra.run.trial122:workunit test osd/bad-inc-map.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh 2026-02-22T23:21:35.734 INFO:tasks.workunit.client.0.trial122.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-22T23:21:35.749 INFO:tasks.workunit.client.0.trial122.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/bad-inc-map 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:8: run: local dir=td/bad-inc-map 2026-02-22T23:21:35.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:9: run: shift 2026-02-22T23:21:35.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: export CEPH_MON=127.0.0.1:59589 2026-02-22T23:21:35.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:11: run: CEPH_MON=127.0.0.1:59589 2026-02-22T23:21:35.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:12: run: export CEPH_ARGS 2026-02-22T23:21:35.751 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: uuidgen 2026-02-22T23:21:35.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:13: run: CEPH_ARGS+='--fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none ' 2026-02-22T23:21:35.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:14: run: CEPH_ARGS+='--mon-host=127.0.0.1:59589 ' 2026-02-22T23:21:35.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:15: run: set -e 2026-02-22T23:21:35.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: set 2026-02-22T23:21:35.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-22T23:21:35.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:17: run: local funcs=TEST_bad_inc_map 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:18: run: for func in $funcs 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:19: run: setup td/bad-inc-map 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/bad-inc-map 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/bad-inc-map 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:21:35.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:21:35.755 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:21:35.755 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:21:35.756 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:21:35.757 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:21:35.757 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:21:35.758 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:21:35.758 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:21:35.758 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:21:35.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:21:35.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:21:35.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:21:35.760 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:21:35.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:21:35.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:21:35.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-22T23:21:35.764 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:21:35.764 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:35.764 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:35.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.12454 2026-02-22T23:21:35.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:21:35.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:21:35.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/bad-inc-map 2026-02-22T23:21:35.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:21:35.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:35.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:35.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.12454 2026-02-22T23:21:35.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:21:35.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-22T23:21:35.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/bad-inc-map 1' TERM HUP INT 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:20: run: TEST_bad_inc_map td/bad-inc-map 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:26: TEST_bad_inc_map: local dir=td/bad-inc-map 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:28: TEST_bad_inc_map: run_mon td/bad-inc-map a 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/bad-inc-map 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/bad-inc-map/a 2026-02-22T23:21:35.770 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/bad-inc-map/a --run-dir=td/bad-inc-map 2026-02-22T23:21:35.800 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:35.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/bad-inc-map/a '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --mon-cluster-log-file=td/bad-inc-map/log --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:21:35.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:21:35.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:21:35.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:21:35.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:21:35.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:21:35.836 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.12454/ceph-mon.a.asok 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:21:35.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.12454/ceph-mon.a.asok config get fsid 2026-02-22T23:21:35.892 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:21:35.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:21:35.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:21:35.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:21:35.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:21:35.894 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:21:35.894 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:21:35.894 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:21:35.894 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:21:35.894 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:35.895 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:35.895 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.12454/ceph-mon.a.asok 2026-02-22T23:21:35.895 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:21:35.895 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.12454/ceph-mon.a.asok config get mon_host 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:29: TEST_bad_inc_map: run_mgr td/bad-inc-map x 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/bad-inc-map 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/bad-inc-map/x 2026-02-22T23:21:35.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:21:36.057 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:36.058 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:21:36.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/bad-inc-map/x '--log-file=td/bad-inc-map/$name.log' '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --run-dir=td/bad-inc-map '--pid-file=td/bad-inc-map/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:21:36.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:30: TEST_bad_inc_map: run_osd td/bad-inc-map 0 2026-02-22T23:21:36.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-22T23:21:36.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/0 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 ' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/0' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/0/journal' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:21:36.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-22T23:21:36.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/0 2026-02-22T23:21:36.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:21:36.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e974ce35-10bb-4d0e-b133-9cfd27aca49f 2026-02-22T23:21:36.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 e974ce35-10bb-4d0e-b133-9cfd27aca49f' 2026-02-22T23:21:36.081 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 e974ce35-10bb-4d0e-b133-9cfd27aca49f 2026-02-22T23:21:36.081 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:21:36.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCAj5tpDsahBRAAI8t77TORtBBE+ob9/u8N1A== 2026-02-22T23:21:36.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCAj5tpDsahBRAAI8t77TORtBBE+ob9/u8N1A=="}' 2026-02-22T23:21:36.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e974ce35-10bb-4d0e-b133-9cfd27aca49f -i td/bad-inc-map/0/new.json 2026-02-22T23:21:36.200 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:21:36.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/0/new.json 2026-02-22T23:21:36.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCAj5tpDsahBRAAI8t77TORtBBE+ob9/u8N1A== --osd-uuid e974ce35-10bb-4d0e-b133-9cfd27aca49f 2026-02-22T23:21:36.224 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:36.223+0000 7f23f2fe08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:36.225 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:36.223+0000 7f23f2fe08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:36.226 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:36.223+0000 7f23f2fe08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:36.227 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:36.223+0000 7f23f2fe08c0 -1 bdev(0x55b73e849c00 td/bad-inc-map/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:21:36.227 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:36.223+0000 7f23f2fe08c0 -1 bluestore(td/bad-inc-map/0) _read_fsid unparsable uuid 2026-02-22T23:21:38.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/0/keyring 2026-02-22T23:21:38.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:21:38.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:21:38.551 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:21:38.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:21:38.662 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:21:38.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:21:38.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/0 --osd-journal=td/bad-inc-map/0/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:21:38.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:21:38.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:21:38.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:21:38.676 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:38.675+0000 7f061b7478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:38.678 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:38.675+0000 7f061b7478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:38.678 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:38.675+0000 7f061b7478c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:38.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:21:38.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:21:38.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:21:38.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:21:38.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:21:38.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:38.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:21:38.768 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:21:38.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:38.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:21:38.962 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:39.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:39.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:39.964 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:21:39.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:21:39.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:39.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:21:40.157 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:40.155+0000 7f061b7478c0 -1 Falling back to public interface 2026-02-22T23:21:40.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:41.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:41.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:41.167 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:21:41.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:21:41.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:41.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:21:41.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:42.370 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:42.370 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:42.371 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:21:42.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:21:42.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:42.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:21:42.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:43.171 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:43.167+0000 7f061b7478c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:21:43.573 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:43.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:43.574 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:21:43.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:21:43.575 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:43.575 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:21:43.769 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2902539249,v1:127.0.0.1:6803/2902539249] [v2:127.0.0.1:6804/2902539249,v1:127.0.0.1:6805/2902539249] exists,up e974ce35-10bb-4d0e-b133-9cfd27aca49f 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:31: TEST_bad_inc_map: run_osd td/bad-inc-map 1 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/1 2026-02-22T23:21:43.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 ' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/1' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/1/journal' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:21:43.773 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:21:43.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:21:43.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:21:43.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:21:43.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:21:43.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/1 2026-02-22T23:21:43.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:21:43.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9f257f09-e808-427d-bdc0-1bac13d5cb76 2026-02-22T23:21:43.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 9f257f09-e808-427d-bdc0-1bac13d5cb76' 2026-02-22T23:21:43.776 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 9f257f09-e808-427d-bdc0-1bac13d5cb76 2026-02-22T23:21:43.776 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:21:43.784 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCHj5tpO+qwLhAA/7051fsn+MYeHJSzfMnddw== 2026-02-22T23:21:43.784 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCHj5tpO+qwLhAA/7051fsn+MYeHJSzfMnddw=="}' 2026-02-22T23:21:43.784 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9f257f09-e808-427d-bdc0-1bac13d5cb76 -i td/bad-inc-map/1/new.json 2026-02-22T23:21:43.983 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:21:43.992 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/1/new.json 2026-02-22T23:21:43.992 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCHj5tpO+qwLhAA/7051fsn+MYeHJSzfMnddw== --osd-uuid 9f257f09-e808-427d-bdc0-1bac13d5cb76 2026-02-22T23:21:44.008 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:44.007+0000 7fd6fb21d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:44.009 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:44.007+0000 7fd6fb21d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:44.010 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:44.007+0000 7fd6fb21d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:44.010 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:44.007+0000 7fd6fb21d8c0 -1 bdev(0x564eac137c00 td/bad-inc-map/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:21:44.010 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:44.007+0000 7fd6fb21d8c0 -1 bluestore(td/bad-inc-map/1) _read_fsid unparsable uuid 2026-02-22T23:21:46.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/1/keyring 2026-02-22T23:21:46.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:21:46.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:21:46.286 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:21:46.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:21:46.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:21:46.554 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:21:46.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/1 --osd-journal=td/bad-inc-map/1/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:21:46.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:21:46.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:21:46.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:21:46.568 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:46.567+0000 7f07bd4fa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:46.570 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:46.567+0000 7f07bd4fa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:46.570 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:46.567+0000 7f07bd4fa8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:46.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:21:46.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:21:46.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:46.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:21:46.951 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:47.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:47.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:47.953 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:21:47.954 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:21:47.954 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:47.955 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:21:48.085 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:48.083+0000 7f07bd4fa8c0 -1 Falling back to public interface 2026-02-22T23:21:48.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:49.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:49.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:49.150 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:21:49.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:21:49.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:49.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:21:49.346 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:50.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:50.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:50.348 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:21:50.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:21:50.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:50.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:21:50.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:51.071 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:51.067+0000 7f07bd4fa8c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:21:51.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:51.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:51.545 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:21:51.545 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:21:51.545 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:51.546 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:21:51.736 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3826110238,v1:127.0.0.1:6811/3826110238] [v2:127.0.0.1:6812/3826110238,v1:127.0.0.1:6813/3826110238] exists,up 9f257f09-e808-427d-bdc0-1bac13d5cb76 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:32: TEST_bad_inc_map: run_osd td/bad-inc-map 2 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/bad-inc-map 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/bad-inc-map/2 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 ' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/bad-inc-map/2' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/bad-inc-map/2/journal' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:21:51.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/bad-inc-map' 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:21:51.738 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/bad-inc-map/$name.log' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/bad-inc-map/$name.pid' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:21:51.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/bad-inc-map/2 2026-02-22T23:21:51.741 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:21:51.742 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9f8d26ab-b3db-42e7-9de8-4d20c760ddf0 2026-02-22T23:21:51.742 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 9f8d26ab-b3db-42e7-9de8-4d20c760ddf0 2026-02-22T23:21:51.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 9f8d26ab-b3db-42e7-9de8-4d20c760ddf0' 2026-02-22T23:21:51.743 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:21:51.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCPj5tp0IDXLBAAnT4wtJbDbxZFtsPdjlnzBA== 2026-02-22T23:21:51.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCPj5tp0IDXLBAAnT4wtJbDbxZFtsPdjlnzBA=="}' 2026-02-22T23:21:51.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9f8d26ab-b3db-42e7-9de8-4d20c760ddf0 -i td/bad-inc-map/2/new.json 2026-02-22T23:21:51.961 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:21:51.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/bad-inc-map/2/new.json 2026-02-22T23:21:51.970 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCPj5tp0IDXLBAAnT4wtJbDbxZFtsPdjlnzBA== --osd-uuid 9f8d26ab-b3db-42e7-9de8-4d20c760ddf0 2026-02-22T23:21:51.985 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:51.983+0000 7f10e0bf38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:51.986 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:51.983+0000 7f10e0bf38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:51.987 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:51.987+0000 7f10e0bf38c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:51.988 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:51.987+0000 7f10e0bf38c0 -1 bdev(0x556efb545c00 td/bad-inc-map/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:21:51.988 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:51.987+0000 7f10e0bf38c0 -1 bluestore(td/bad-inc-map/2) _read_fsid unparsable uuid 2026-02-22T23:21:54.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/bad-inc-map/2/keyring 2026-02-22T23:21:54.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:21:54.274 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:21:54.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:21:54.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/bad-inc-map/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:21:54.543 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:21:54.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:21:54.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=54b2babe-126d-4ec1-a02f-a604c256e846 --auth-supported=none --mon-host=127.0.0.1:59589 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/bad-inc-map/2 --osd-journal=td/bad-inc-map/2/journal --chdir= --run-dir=td/bad-inc-map '--admin-socket=/tmp/ceph-asok.12454/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/bad-inc-map/$name.log' '--pid-file=td/bad-inc-map/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:21:54.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:21:54.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:21:54.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:21:54.564 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:54.563+0000 7f6fcaafc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:54.566 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:54.563+0000 7f6fcaafc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:54.567 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:54.563+0000 7f6fcaafc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:21:54.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:21:54.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:21:54.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:21:54.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:21:54.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:21:54.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:54.735 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:21:54.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:21:54.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:54.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:21:54.929 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:55.930 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:55.930 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:55.931 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:21:55.931 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:21:55.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:55.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:21:56.127 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:56.281 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:56.279+0000 7f6fcaafc8c0 -1 Falling back to public interface 2026-02-22T23:21:57.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:57.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:57.129 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:21:57.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:21:57.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:57.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:21:57.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:58.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:58.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:58.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:21:58.327 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:21:58.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:58.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:21:58.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:21:59.265 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:21:59.263+0000 7f6fcaafc8c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:21:59.524 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:21:59.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:21:59.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:21:59.525 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:21:59.526 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:21:59.526 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:21:59.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:00.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:00.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:00.726 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:22:00.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:22:00.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:00.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:00.920 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4096920116,v1:127.0.0.1:6819/4096920116] [v2:127.0.0.1:6820/4096920116,v1:127.0.0.1:6821/4096920116] exists,up 9f8d26ab-b3db-42e7-9de8-4d20c760ddf0 2026-02-22T23:22:00.920 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:00.921 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:00.921 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:00.921 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:34: TEST_bad_inc_map: ceph config set osd.2 osd_inject_bad_map_crc_probability 1 2026-02-22T23:22:01.123 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:37: TEST_bad_inc_map: create_pool foo 8 2026-02-22T23:22:01.123 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 8 2026-02-22T23:22:01.401 INFO:tasks.workunit.client.0.trial122.stderr:pool 'foo' already exists 2026-02-22T23:22:01.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:22:02.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:38: TEST_bad_inc_map: ceph osd pool set foo min_size 1 2026-02-22T23:22:02.716 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 min_size to 1 2026-02-22T23:22:02.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:39: TEST_bad_inc_map: ceph osd pool set foo min_size 2 2026-02-22T23:22:03.017 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 min_size to 2 2026-02-22T23:22:03.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:41: TEST_bad_inc_map: sleep 5 2026-02-22T23:22:08.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: TIMEOUT=10 2026-02-22T23:22:08.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:44: TEST_bad_inc_map: wait_for_osd up 0 2026-02-22T23:22:08.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:22:08.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:22:08.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:22:08.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:22:08.035 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:08.036 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-22T23:22:08.036 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:22:08.036 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:08.036 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:08.231 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2902539249,v1:127.0.0.1:6803/2902539249] [v2:127.0.0.1:6804/2902539249,v1:127.0.0.1:6805/2902539249] exists,up e974ce35-10bb-4d0e-b133-9cfd27aca49f 2026-02-22T23:22:08.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:08.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:08.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: TIMEOUT=10 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:45: TEST_bad_inc_map: wait_for_osd up 1 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:22:08.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-22T23:22:08.235 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:08.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:22:08.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:08.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:22:08.425 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3826110238,v1:127.0.0.1:6811/3826110238] [v2:127.0.0.1:6812/3826110238,v1:127.0.0.1:6813/3826110238] exists,up 9f257f09-e808-427d-bdc0-1bac13d5cb76 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: TIMEOUT=10 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:46: TEST_bad_inc_map: wait_for_osd up 2 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 10 )) 2026-02-22T23:22:08.426 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:08.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:22:08.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:08.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:08.620 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 13 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4096920116,v1:127.0.0.1:6819/4096920116] [v2:127.0.0.1:6820/4096920116,v1:127.0.0.1:6821/4096920116] exists,up 9f8d26ab-b3db-42e7-9de8-4d20c760ddf0 2026-02-22T23:22:08.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:08.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:08.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:08.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:49: TEST_bad_inc_map: grep 'injecting map crc failure' td/bad-inc-map/osd.2.log 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:01.347+0000 7f6fba155640 -1 osd.2 13 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:01.447+0000 7f6fba155640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:01.447+0000 7f6fba155640 -1 osd.2 14 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:02.611+0000 7f6fba155640 -1 osd.2 15 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:02.715+0000 7f6fba155640 -1 osd.2 17 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:02.915+0000 7f6fba155640 -1 osd.2 18 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:03.015+0000 7f6fba155640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:03.015+0000 7f6fb8151640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:03.019+0000 7f6fb8151640 -1 osd.2 19 handle_osd_map injecting map crc failure 2026-02-22T23:22:08.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:50: TEST_bad_inc_map: grep 'bailing because last' td/bad-inc-map/osd.2.log 2026-02-22T23:22:08.623 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:01.347+0000 7f6fba155640 10 osd.2 13 handle_osd_map bailing because last < start (13<14) 2026-02-22T23:22:08.623 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:01.447+0000 7f6fba155640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-22T23:22:08.623 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:01.447+0000 7f6fba155640 10 osd.2 14 handle_osd_map bailing because last < start (14<15) 2026-02-22T23:22:08.623 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:02.611+0000 7f6fba155640 10 osd.2 15 handle_osd_map bailing because last < start (15<16) 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:02.715+0000 7f6fba155640 10 osd.2 17 handle_osd_map bailing because last < start (17<18) 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:02.915+0000 7f6fba155640 10 osd.2 18 handle_osd_map bailing because last < start (18<19) 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:03.015+0000 7f6fba155640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:03.015+0000 7f6fb8151640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:22:03.019+0000 7f6fb8151640 10 osd.2 19 handle_osd_map bailing because last < start (19<20) 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stdout:success 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:52: TEST_bad_inc_map: echo success 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:54: TEST_bad_inc_map: delete_pool foo 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=foo 2026-02-22T23:22:08.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete foo foo --yes-i-really-really-mean-it 2026-02-22T23:22:08.867 INFO:tasks.workunit.client.0.trial122.stderr:pool 'foo' does not exist 2026-02-22T23:22:08.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:55: TEST_bad_inc_map: kill_daemons td/bad-inc-map 2026-02-22T23:22:08.876 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:22:08.876 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:22:08.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:22:08.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:22:08.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:22:17.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:22:17.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/bad-inc-map.sh:21: run: teardown td/bad-inc-map 2026-02-22T23:22:17.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-22T23:22:17.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:22:17.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-22T23:22:17.193 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:22:17.193 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:22:17.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:22:17.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:22:17.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:22:17.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:22:17.199 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:22:17.200 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:22:17.200 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:22:17.202 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:22:17.202 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:22:17.202 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:22:17.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:22:17.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:22:17.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:22:17.203 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:22:17.204 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:22:17.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:22:17.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-22T23:22:17.215 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:22:17.215 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.215 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:22:17.216 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.12454 2026-02-22T23:22:17.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:22:17.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/bad-inc-map 0 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/bad-inc-map 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/bad-inc-map KILL 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:22:17.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:22:17.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:22:17.220 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:22:17.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:22:17.221 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:22:17.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:22:17.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:22:17.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:22:17.223 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:22:17.223 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:22:17.224 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:22:17.224 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:22:17.225 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:22:17.225 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-22T23:22:17.226 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/bad-inc-map 2026-02-22T23:22:17.227 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:22:17.227 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.227 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.12454 2026-02-22T23:22:17.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.12454 2026-02-22T23:22:17.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:22:17.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:22:17.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-22T23:22:17.229 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-22T23:22:17.230 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-22T23:22:17.283 INFO:tasks.workunit:Running workunit osd/divergent-priors.sh... 2026-02-22T23:22:17.284 DEBUG:teuthology.orchestra.run.trial122:workunit test osd/divergent-priors.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh 2026-02-22T23:22:17.337 INFO:tasks.workunit.client.0.trial122.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-22T23:22:17.342 INFO:tasks.workunit.client.0.trial122.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-22T23:22:17.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:22:17.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:22:17.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-22T23:22:17.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/divergent-priors 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:21: run: local dir=td/divergent-priors 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:22: run: shift 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: export loglen=12 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:25: run: loglen=12 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: export divisor=3 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:26: run: divisor=3 2026-02-22T23:22:17.343 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: expr 12 / 2 2026-02-22T23:22:17.344 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: export trim=6 2026-02-22T23:22:17.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:27: run: trim=6 2026-02-22T23:22:17.345 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: expr 6 / 3 2026-02-22T23:22:17.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: export DIVERGENT_WRITE=2 2026-02-22T23:22:17.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:28: run: DIVERGENT_WRITE=2 2026-02-22T23:22:17.345 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: expr 6 / 3 2026-02-22T23:22:17.346 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: export DIVERGENT_REMOVE=2 2026-02-22T23:22:17.346 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:29: run: DIVERGENT_REMOVE=2 2026-02-22T23:22:17.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: expr 6 / 3 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: export DIVERGENT_CREATE=2 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:30: run: DIVERGENT_CREATE=2 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: export poolname=test 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:31: run: poolname=test 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: export testobjects=100 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:32: run: testobjects=100 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: export CEPH_MON=127.0.0.1:7115 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:34: run: CEPH_MON=127.0.0.1:7115 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:35: run: export CEPH_ARGS 2026-02-22T23:22:17.348 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: uuidgen 2026-02-22T23:22:17.349 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:36: run: CEPH_ARGS+='--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none ' 2026-02-22T23:22:17.349 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:37: run: CEPH_ARGS+='--mon-host=127.0.0.1:7115 ' 2026-02-22T23:22:17.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:39: run: CEPH_ARGS+='--osd_force_auth_primary_missing_objects=1000000 ' 2026-02-22T23:22:17.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:40: run: CEPH_ARGS+='--osd_debug_pg_log_writeout=true ' 2026-02-22T23:22:17.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:41: run: CEPH_ARGS+='--osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:22:17.350 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: set 2026-02-22T23:22:17.350 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-22T23:22:17.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:43: run: local 'funcs=TEST_divergent 2026-02-22T23:22:17.352 INFO:tasks.workunit.client.0.trial122.stderr:TEST_divergent_2 2026-02-22T23:22:17.352 INFO:tasks.workunit.client.0.trial122.stderr:TEST_divergent_3 2026-02-22T23:22:17.352 INFO:tasks.workunit.client.0.trial122.stderr:TEST_divergent_ec' 2026-02-22T23:22:17.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-22T23:22:17.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:22:17.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:22:17.354 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:22:17.354 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:22:17.355 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:22:17.356 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:22:17.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:22:17.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:22:17.357 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:22:17.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:22:17.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:22:17.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:22:17.359 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:22:17.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:22:17.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:22:17.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:22:17.362 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:22:17.362 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.363 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:22:17.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:22:17.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:22:17.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-22T23:22:17.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:22:17.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20359 2026-02-22T23:22:17.367 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent td/divergent-priors 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:57: TEST_divergent: local dir=td/divergent-priors 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:60: TEST_divergent: local dummyfile=/etc/fstab 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:61: TEST_divergent: local dummyfile2=/etc/resolv.conf 2026-02-22T23:22:17.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:63: TEST_divergent: local num_osds=3 2026-02-22T23:22:17.370 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: expr 3 - 1 2026-02-22T23:22:17.370 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: seq 0 2 2026-02-22T23:22:17.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:64: TEST_divergent: local 'osds=0 2026-02-22T23:22:17.371 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:22:17.371 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:22:17.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:65: TEST_divergent: run_mon td/divergent-priors a 2026-02-22T23:22:17.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-22T23:22:17.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:22:17.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:22:17.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:22:17.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-22T23:22:17.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-22T23:22:17.393 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:17.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:22:17.422 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:22:17.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:22:17.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:22:17.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:22:17.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:22:17.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:22:17.423 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:22:17.424 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get fsid 2026-02-22T23:22:17.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:22:17.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:22:17.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:22:17.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:22:17.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:22:17.479 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:22:17.479 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:22:17.479 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:22:17.480 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:22:17.480 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.480 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.480 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:22:17.480 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:22:17.481 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get mon_host 2026-02-22T23:22:17.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:66: TEST_divergent: run_mgr td/divergent-priors x 2026-02-22T23:22:17.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-22T23:22:17.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:22:17.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:22:17.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:22:17.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-22T23:22:17.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:22:17.646 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:22:17.647 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:22:17.647 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:22:17.647 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:22:17.647 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.647 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.647 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:17.648 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:22:17.648 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:22:17.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-22T23:22:17.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 0 2026-02-22T23:22:17.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:22:17.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:22:17.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:22:17.672 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:22:17.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:22:17.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:22:17.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=9f76a466-b535-45aa-bf72-3bfe89505d1c 2026-02-22T23:22:17.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 9f76a466-b535-45aa-bf72-3bfe89505d1c' 2026-02-22T23:22:17.675 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 9f76a466-b535-45aa-bf72-3bfe89505d1c 2026-02-22T23:22:17.676 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:22:17.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCpj5tpHVDZKBAAH3ncGWHW1zE+tcrunlx7mw== 2026-02-22T23:22:17.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCpj5tpHVDZKBAAH3ncGWHW1zE+tcrunlx7mw=="}' 2026-02-22T23:22:17.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 9f76a466-b535-45aa-bf72-3bfe89505d1c -i td/divergent-priors/0/new.json 2026-02-22T23:22:17.798 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:17.807 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-22T23:22:17.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCpj5tpHVDZKBAAH3ncGWHW1zE+tcrunlx7mw== --osd-uuid 9f76a466-b535-45aa-bf72-3bfe89505d1c 2026-02-22T23:22:17.827 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:17.822+0000 7f3bc384f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:17.828 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:17.826+0000 7f3bc384f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:17.829 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:17.826+0000 7f3bc384f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:17.829 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:17.826+0000 7f3bc384f8c0 -1 bdev(0x55784f8c3c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:22:17.829 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:17.826+0000 7f3bc384f8c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-22T23:22:20.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-22T23:22:20.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:22:20.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:22:20.156 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:22:20.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:22:20.426 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:22:20.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:22:20.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:22:20.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:22:20.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:22:20.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:22:20.444 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:20.442+0000 7f12fc90c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:20.445 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:20.442+0000 7f12fc90c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:20.446 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:20.442+0000 7f12fc90c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:20.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:22:20.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:22:20.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:22:20.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:22:20.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:22:20.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:20.629 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:20.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:22:20.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:20.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:20.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:21.831 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:21.831 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:21.831 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:22:21.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:22:21.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:21.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:22.033 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:22.185 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:22.182+0000 7f12fc90c8c0 -1 Falling back to public interface 2026-02-22T23:22:23.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:23.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:23.034 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:22:23.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:22:23.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:23.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:23.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:24.229 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:24.230 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:24.230 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:22:24.230 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:22:24.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:24.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:24.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:25.200 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:25.198+0000 7f12fc90c8c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:22:25.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:25.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:25.426 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:22:25.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:22:25.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:25.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:25.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:26.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:26.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:26.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:22:26.623 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:22:26.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:26.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:22:26.819 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3764696768,v1:127.0.0.1:6803/3764696768] [v2:127.0.0.1:6804/3764696768,v1:127.0.0.1:6805/3764696768] exists,up 9f76a466-b535-45aa-bf72-3bfe89505d1c 2026-02-22T23:22:26.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 1 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:22:26.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:22:26.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:22:26.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:22:26.823 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fe00adae-737d-4f7c-aec2-9f44e8f9487e 2026-02-22T23:22:26.823 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 fe00adae-737d-4f7c-aec2-9f44e8f9487e' 2026-02-22T23:22:26.823 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 fe00adae-737d-4f7c-aec2-9f44e8f9487e 2026-02-22T23:22:26.823 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:22:26.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCyj5tptBzZMRAAxdBtK1qb/tcuc0oTF8D2nA== 2026-02-22T23:22:26.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCyj5tptBzZMRAAxdBtK1qb/tcuc0oTF8D2nA=="}' 2026-02-22T23:22:26.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fe00adae-737d-4f7c-aec2-9f44e8f9487e -i td/divergent-priors/1/new.json 2026-02-22T23:22:27.046 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:22:27.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-22T23:22:27.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCyj5tptBzZMRAAxdBtK1qb/tcuc0oTF8D2nA== --osd-uuid fe00adae-737d-4f7c-aec2-9f44e8f9487e 2026-02-22T23:22:27.074 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:27.070+0000 7f174cce28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:27.076 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:27.074+0000 7f174cce28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:27.077 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:27.074+0000 7f174cce28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:27.077 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:27.074+0000 7f174cce28c0 -1 bdev(0x55d815413c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:22:27.077 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:27.074+0000 7f174cce28c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-22T23:22:29.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-22T23:22:29.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:22:29.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:22:29.354 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:22:29.354 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:22:29.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:22:29.630 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:22:29.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:22:29.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:22:29.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:22:29.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:22:29.648 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:29.646+0000 7f43fcfb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:29.649 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:29.646+0000 7f43fcfb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:29.650 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:29.646+0000 7f43fcfb28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:29.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:22:29.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:22:29.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:22:29.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:22:29.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:22:29.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:29.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:22:29.828 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:29.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:29.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:22:30.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:31.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:31.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:31.024 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:22:31.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:22:31.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:31.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:22:31.169 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:31.166+0000 7f43fcfb28c0 -1 Falling back to public interface 2026-02-22T23:22:31.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:32.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:32.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:32.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:22:32.220 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:22:32.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:32.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:22:32.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:33.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:33.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:33.416 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:22:33.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:22:33.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:33.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:22:33.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:34.167 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:34.162+0000 7f43fcfb28c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:22:34.620 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:34.620 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:34.621 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:22:34.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:22:34.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:34.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:22:34.818 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3737256882,v1:127.0.0.1:6811/3737256882] [v2:127.0.0.1:6812/3737256882,v1:127.0.0.1:6813/3737256882] exists,up fe00adae-737d-4f7c-aec2-9f44e8f9487e 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:67: TEST_divergent: for i in $osds 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:69: TEST_divergent: run_osd td/divergent-priors 2 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:22:34.819 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:22:34.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:22:34.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:22:34.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:22:34.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:22:34.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:22:34.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:22:34.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=6ecf0020-a7e1-44bf-9f24-2a5230b4e516 2026-02-22T23:22:34.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 6ecf0020-a7e1-44bf-9f24-2a5230b4e516' 2026-02-22T23:22:34.822 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 6ecf0020-a7e1-44bf-9f24-2a5230b4e516 2026-02-22T23:22:34.823 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:22:34.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQC6j5tpnxrQMRAADIqeOdUj6glvL3qZDde64g== 2026-02-22T23:22:34.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQC6j5tpnxrQMRAADIqeOdUj6glvL3qZDde64g=="}' 2026-02-22T23:22:34.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 6ecf0020-a7e1-44bf-9f24-2a5230b4e516 -i td/divergent-priors/2/new.json 2026-02-22T23:22:35.037 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:22:35.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-22T23:22:35.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC6j5tpnxrQMRAADIqeOdUj6glvL3qZDde64g== --osd-uuid 6ecf0020-a7e1-44bf-9f24-2a5230b4e516 2026-02-22T23:22:35.064 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:35.062+0000 7fe51ad498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:35.066 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:35.062+0000 7fe51ad498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:35.067 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:35.062+0000 7fe51ad498c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:35.067 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:35.066+0000 7fe51ad498c0 -1 bdev(0x560d76bc5c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:22:35.067 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:35.066+0000 7fe51ad498c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-22T23:22:37.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-22T23:22:37.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:22:37.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:22:37.361 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:22:37.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:22:37.629 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:22:37.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:22:37.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:22:37.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:22:37.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:22:37.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:22:37.656 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:37.654+0000 7f933b8bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:37.657 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:37.654+0000 7f933b8bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:37.659 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:37.654+0000 7f933b8bc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:22:37.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:22:37.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:22:37.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:22:37.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:22:37.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:22:37.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:37.828 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:22:37.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:22:37.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:37.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:38.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:39.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:39.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:39.025 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:22:39.025 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:22:39.025 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:39.025 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:39.161 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:39.158+0000 7f933b8bc8c0 -1 Falling back to public interface 2026-02-22T23:22:39.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:40.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:40.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:40.222 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:22:40.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:22:40.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:40.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:40.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:41.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:41.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:41.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:22:41.416 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:22:41.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:41.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:41.611 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:42.217 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:22:42.214+0000 7f933b8bc8c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:22:42.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:42.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:42.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:22:42.612 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:22:42.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:42.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:42.814 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:22:43.816 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:22:43.816 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:22:43.817 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:22:43.817 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:22:43.817 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:22:43.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:22:44.015 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/272057748,v1:127.0.0.1:6819/272057748] [v2:127.0.0.1:6820/272057748,v1:127.0.0.1:6821/272057748] exists,up 6ecf0020-a7e1-44bf-9f24-2a5230b4e516 2026-02-22T23:22:44.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:22:44.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:22:44.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:22:44.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:72: TEST_divergent: ceph osd set noout 2026-02-22T23:22:44.310 INFO:tasks.workunit.client.0.trial122.stderr:noout is set 2026-02-22T23:22:44.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:73: TEST_divergent: ceph osd set noin 2026-02-22T23:22:44.613 INFO:tasks.workunit.client.0.trial122.stderr:noin is set 2026-02-22T23:22:44.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:74: TEST_divergent: ceph osd set nodown 2026-02-22T23:22:44.919 INFO:tasks.workunit.client.0.trial122.stderr:nodown is set 2026-02-22T23:22:44.930 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:75: TEST_divergent: create_pool test 1 1 2026-02-22T23:22:44.930 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-22T23:22:45.167 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' already exists 2026-02-22T23:22:45.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:22:46.178 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:76: TEST_divergent: ceph osd pool set test size 3 2026-02-22T23:22:46.471 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 size to 3 2026-02-22T23:22:46.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:77: TEST_divergent: ceph osd pool set test min_size 2 2026-02-22T23:22:46.777 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 min_size to 2 2026-02-22T23:22:46.786 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:79: TEST_divergent: flush_pg_stats 2026-02-22T23:22:46.786 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:22:46.787 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:22:46.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:22:46.989 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:22:46.989 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:22:46.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:22:46.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:46.990 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:22:47.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-22T23:22:47.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-22T23:22:47.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-22T23:22:47.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:47.071 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:22:47.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-22T23:22:47.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-22T23:22:47.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-22T23:22:47.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:47.152 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:22:47.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-22T23:22:47.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-22T23:22:47.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574851' 2026-02-22T23:22:47.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:47.232 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-22T23:22:47.232 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:47.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:22:47.234 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-22T23:22:47.234 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:47.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-22T23:22:47.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-22T23:22:47.236 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836487 2026-02-22T23:22:47.236 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:22:47.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836487 2026-02-22T23:22:47.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:22:48.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:22:48.444 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:22:48.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-22T23:22:48.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:48.641 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-22T23:22:48.641 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:48.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:22:48.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-22T23:22:48.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:48.645 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-22T23:22:48.645 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705669 2026-02-22T23:22:48.645 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-22T23:22:48.645 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:22:48.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-22T23:22:48.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:48.858 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-22T23:22:48.858 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:48.859 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:22:48.860 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-22T23:22:48.860 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:48.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-22T23:22:48.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-22T23:22:48.862 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 55834574851 2026-02-22T23:22:48.862 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:22:49.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-22T23:22:49.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:80: TEST_divergent: wait_for_clean 2026-02-22T23:22:49.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:22:49.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:22:49.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:22:49.067 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:22:49.067 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:22:49.067 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:22:49.067 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:22:49.067 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:22:49.067 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:22:49.159 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:22:49.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:22:49.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:22:49.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:22:49.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:22:49.160 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:22:49.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:22:49.359 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:22:49.359 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:22:49.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:22:49.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:49.360 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:22:49.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-22T23:22:49.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-22T23:22:49.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-22T23:22:49.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:49.437 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:22:49.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-22T23:22:49.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-22T23:22:49.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671' 2026-02-22T23:22:49.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:49.517 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:22:49.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-22T23:22:49.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-22T23:22:49.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574854' 2026-02-22T23:22:49.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:49.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-22T23:22:49.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:49.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:22:49.596 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-22T23:22:49.596 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:49.597 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-22T23:22:49.597 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-22T23:22:49.598 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836489 2026-02-22T23:22:49.598 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:22:49.797 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-22T23:22:49.797 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:22:50.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:22:50.799 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:22:50.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-22T23:22:50.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:50.999 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-22T23:22:51.000 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:51.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:22:51.001 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-22T23:22:51.002 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:51.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-22T23:22:51.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-22T23:22:51.003 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705671 2026-02-22T23:22:51.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:22:51.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-22T23:22:51.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:51.209 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574854 2026-02-22T23:22:51.209 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:51.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:22:51.211 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574854 2026-02-22T23:22:51.211 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:51.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-22T23:22:51.214 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 55834574854 2026-02-22T23:22:51.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-22T23:22:51.214 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:22:51.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-22T23:22:51.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:22:51.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:22:51.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:22:51.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:22:51.675 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:22:51.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:22:51.868 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:22:51.868 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:22:51.868 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:22:52.126 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:22:52.126 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:22:52.126 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:22:52.127 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: ceph pg dump pgs --format=json 2026-02-22T23:22:52.127 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: jq '.pg_stats[0].up_primary' 2026-02-22T23:22:52.306 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:22:52.315 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:83: TEST_divergent: local divergent=1 2026-02-22T23:22:52.316 INFO:tasks.workunit.client.0.trial122.stdout:primary and soon to be divergent is 1 2026-02-22T23:22:52.316 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:84: TEST_divergent: echo 'primary and soon to be divergent is 1' 2026-02-22T23:22:52.316 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:85: TEST_divergent: ceph pg dump pgs 2026-02-22T23:22:52.505 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:22:52.505 INFO:tasks.workunit.client.0.trial122.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-22T23:22:45.270941+0000 0'0 26:23 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:35:30.942054+0000 0 0 2026-02-22T23:22:52.505 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:22:52.505 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:22:52.506 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:22:52.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:86: TEST_divergent: local non_divergent= 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 0 = 1 ']' 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0' 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 1 = 1 ']' 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:90: TEST_divergent: continue 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:87: TEST_divergent: for i in $osds 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:89: TEST_divergent: '[' 2 = 1 ']' 2026-02-22T23:22:52.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:92: TEST_divergent: non_divergent=' 0 2' 2026-02-22T23:22:52.516 INFO:tasks.workunit.client.0.trial122.stdout:writing initial objects 2026-02-22T23:22:52.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:95: TEST_divergent: echo 'writing initial objects' 2026-02-22T23:22:52.516 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: seq 1 100 2026-02-22T23:22:52.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_1 /etc/fstab 2026-02-22T23:22:52.538 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.538 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_2 /etc/fstab 2026-02-22T23:22:52.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_3 /etc/fstab 2026-02-22T23:22:52.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_4 /etc/fstab 2026-02-22T23:22:52.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_5 /etc/fstab 2026-02-22T23:22:52.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_6 /etc/fstab 2026-02-22T23:22:52.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_7 /etc/fstab 2026-02-22T23:22:52.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.708 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_8 /etc/fstab 2026-02-22T23:22:52.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_9 /etc/fstab 2026-02-22T23:22:52.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_10 /etc/fstab 2026-02-22T23:22:52.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_11 /etc/fstab 2026-02-22T23:22:52.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_12 /etc/fstab 2026-02-22T23:22:52.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_13 /etc/fstab 2026-02-22T23:22:52.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_14 /etc/fstab 2026-02-22T23:22:52.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_15 /etc/fstab 2026-02-22T23:22:52.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_16 /etc/fstab 2026-02-22T23:22:52.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_17 /etc/fstab 2026-02-22T23:22:52.979 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:52.979 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_18 /etc/fstab 2026-02-22T23:22:53.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_19 /etc/fstab 2026-02-22T23:22:53.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.034 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_20 /etc/fstab 2026-02-22T23:22:53.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_21 /etc/fstab 2026-02-22T23:22:53.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_22 /etc/fstab 2026-02-22T23:22:53.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_23 /etc/fstab 2026-02-22T23:22:53.137 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.137 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_24 /etc/fstab 2026-02-22T23:22:53.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_25 /etc/fstab 2026-02-22T23:22:53.187 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.187 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_26 /etc/fstab 2026-02-22T23:22:53.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_27 /etc/fstab 2026-02-22T23:22:53.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_28 /etc/fstab 2026-02-22T23:22:53.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_29 /etc/fstab 2026-02-22T23:22:53.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_30 /etc/fstab 2026-02-22T23:22:53.324 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.324 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_31 /etc/fstab 2026-02-22T23:22:53.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_32 /etc/fstab 2026-02-22T23:22:53.380 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.380 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_33 /etc/fstab 2026-02-22T23:22:53.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_34 /etc/fstab 2026-02-22T23:22:53.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_35 /etc/fstab 2026-02-22T23:22:53.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_36 /etc/fstab 2026-02-22T23:22:53.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_37 /etc/fstab 2026-02-22T23:22:53.520 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.520 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_38 /etc/fstab 2026-02-22T23:22:53.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_39 /etc/fstab 2026-02-22T23:22:53.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_40 /etc/fstab 2026-02-22T23:22:53.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_41 /etc/fstab 2026-02-22T23:22:53.632 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.632 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_42 /etc/fstab 2026-02-22T23:22:53.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.661 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_43 /etc/fstab 2026-02-22T23:22:53.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_44 /etc/fstab 2026-02-22T23:22:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_45 /etc/fstab 2026-02-22T23:22:53.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_46 /etc/fstab 2026-02-22T23:22:53.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_47 /etc/fstab 2026-02-22T23:22:53.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_48 /etc/fstab 2026-02-22T23:22:53.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_49 /etc/fstab 2026-02-22T23:22:53.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_50 /etc/fstab 2026-02-22T23:22:53.884 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_51 /etc/fstab 2026-02-22T23:22:53.913 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.913 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_52 /etc/fstab 2026-02-22T23:22:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_53 /etc/fstab 2026-02-22T23:22:53.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_54 /etc/fstab 2026-02-22T23:22:53.996 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:53.996 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_55 /etc/fstab 2026-02-22T23:22:54.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_56 /etc/fstab 2026-02-22T23:22:54.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_57 /etc/fstab 2026-02-22T23:22:54.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_58 /etc/fstab 2026-02-22T23:22:54.108 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.108 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_59 /etc/fstab 2026-02-22T23:22:54.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_60 /etc/fstab 2026-02-22T23:22:54.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_61 /etc/fstab 2026-02-22T23:22:54.190 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.190 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_62 /etc/fstab 2026-02-22T23:22:54.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_63 /etc/fstab 2026-02-22T23:22:54.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_64 /etc/fstab 2026-02-22T23:22:54.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_65 /etc/fstab 2026-02-22T23:22:54.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_66 /etc/fstab 2026-02-22T23:22:54.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_67 /etc/fstab 2026-02-22T23:22:54.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_68 /etc/fstab 2026-02-22T23:22:54.380 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.380 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_69 /etc/fstab 2026-02-22T23:22:54.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_70 /etc/fstab 2026-02-22T23:22:54.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_71 /etc/fstab 2026-02-22T23:22:54.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_72 /etc/fstab 2026-02-22T23:22:54.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_73 /etc/fstab 2026-02-22T23:22:54.520 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.520 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_74 /etc/fstab 2026-02-22T23:22:54.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_75 /etc/fstab 2026-02-22T23:22:54.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_76 /etc/fstab 2026-02-22T23:22:54.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_77 /etc/fstab 2026-02-22T23:22:54.632 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.632 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_78 /etc/fstab 2026-02-22T23:22:54.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_79 /etc/fstab 2026-02-22T23:22:54.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_80 /etc/fstab 2026-02-22T23:22:54.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_81 /etc/fstab 2026-02-22T23:22:54.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_82 /etc/fstab 2026-02-22T23:22:54.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_83 /etc/fstab 2026-02-22T23:22:54.793 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.793 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_84 /etc/fstab 2026-02-22T23:22:54.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_85 /etc/fstab 2026-02-22T23:22:54.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_86 /etc/fstab 2026-02-22T23:22:54.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_87 /etc/fstab 2026-02-22T23:22:54.901 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.901 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_88 /etc/fstab 2026-02-22T23:22:54.929 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.929 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_89 /etc/fstab 2026-02-22T23:22:54.957 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.957 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_90 /etc/fstab 2026-02-22T23:22:54.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:54.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_91 /etc/fstab 2026-02-22T23:22:55.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_92 /etc/fstab 2026-02-22T23:22:55.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_93 /etc/fstab 2026-02-22T23:22:55.069 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.069 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_94 /etc/fstab 2026-02-22T23:22:55.096 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_95 /etc/fstab 2026-02-22T23:22:55.124 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.124 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_96 /etc/fstab 2026-02-22T23:22:55.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_97 /etc/fstab 2026-02-22T23:22:55.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_98 /etc/fstab 2026-02-22T23:22:55.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_99 /etc/fstab 2026-02-22T23:22:55.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:97: TEST_divergent: for i in $(seq 1 $testobjects) 2026-02-22T23:22:55.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:99: TEST_divergent: rados -p test put existing_100 /etc/fstab 2026-02-22T23:22:55.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:22:55.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:102: TEST_divergent: wait_for_clean 2026-02-22T23:22:55.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:22:55.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:22:55.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:22:55.265 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:22:55.266 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:22:55.266 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:22:55.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:22:55.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:22:55.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:22:55.333 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:22:55.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:22:55.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:22:55.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:22:55.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:22:55.334 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:22:55.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:22:55.532 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:22:55.532 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:22:55.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:22:55.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:55.532 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:22:55.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-22T23:22:55.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-22T23:22:55.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-22T23:22:55.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:55.612 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:22:55.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-22T23:22:55.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-22T23:22:55.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675' 2026-02-22T23:22:55.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:22:55.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:22:55.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-22T23:22:55.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-22T23:22:55.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574857' 2026-02-22T23:22:55.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:55.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-22T23:22:55.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:55.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:22:55.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-22T23:22:55.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:55.777 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-22T23:22:55.777 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-22T23:22:55.777 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836492 2026-02-22T23:22:55.777 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:22:55.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-22T23:22:55.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:22:56.976 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:22:56.977 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:22:57.185 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-22T23:22:57.185 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:57.186 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-22T23:22:57.186 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:57.187 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:22:57.188 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-22T23:22:57.188 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:57.189 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-22T23:22:57.190 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-22T23:22:57.190 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705675 2026-02-22T23:22:57.190 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:22:57.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-22T23:22:57.401 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:22:57.401 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-22T23:22:57.402 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:22:57.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:22:57.403 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-22T23:22:57.403 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:22:57.405 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-22T23:22:57.405 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-22T23:22:57.405 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 55834574857 2026-02-22T23:22:57.406 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:22:57.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-22T23:22:57.618 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:22:57.619 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:22:57.619 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:22:57.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:22:57.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:22:57.894 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:22:57.894 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:22:57.894 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:22:57.894 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:22:57.894 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:22:57.894 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:22:58.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:22:58.096 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:22:58.096 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:22:58.096 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:22:58.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:22:58.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:22:58.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:22:58.361 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: get_pg test existing_1 2026-02-22T23:22:58.361 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-22T23:22:58.361 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-22T23:22:58.361 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-22T23:22:58.361 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-22T23:22:58.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:104: TEST_divergent: local pgid=1.0 2026-02-22T23:22:58.568 INFO:tasks.workunit.client.0.trial122.stdout:blackholing osds 0 2 2026-02-22T23:22:58.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:107: TEST_divergent: echo 'blackholing osds 0 2' 2026-02-22T23:22:58.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:108: TEST_divergent: ceph pg dump pgs 2026-02-22T23:22:58.757 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:22:58.757 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-22T23:22:45.270941+0000 29'100 29:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:35:30.942054+0000 0 0 2026-02-22T23:22:58.757 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:22:58.758 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:22:58.758 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:22:58.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2026-02-22T23:22:58.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.0 2026-02-22T23:22:58.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:22:58.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:22:58.768 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:22:58.768 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:58.768 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:58.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:22:58.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2026-02-22T23:22:58.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-22T23:22:58.820 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:22:58.821 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:22:58.821 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:22:58.826 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:109: TEST_divergent: for i in $non_divergent 2026-02-22T23:22:58.827 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: get_asok_path osd.2 2026-02-22T23:22:58.827 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:22:58.827 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:22:58.827 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:22:58.827 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:58.827 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:58.828 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:22:58.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: CEPH_ARGS= 2026-02-22T23:22:58.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:111: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-22T23:22:58.879 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:22:58.879 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:22:58.879 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:22:58.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:114: TEST_divergent: local case5=100 2026-02-22T23:22:58.885 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: expr 100 - 1 2026-02-22T23:22:58.886 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:115: TEST_divergent: local case3=99 2026-02-22T23:22:58.886 INFO:tasks.workunit.client.0.trial122.stdout:writing divergent object 2026-02-22T23:22:58.887 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:117: TEST_divergent: echo 'writing divergent object' 2026-02-22T23:22:58.887 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:119: TEST_divergent: echo 'create missing divergent object' 2026-02-22T23:22:58.887 INFO:tasks.workunit.client.0.trial122.stdout:create missing divergent object 2026-02-22T23:22:58.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:120: TEST_divergent: inject_eio rep data test existing_99 td/divergent-priors 0 2026-02-22T23:22:58.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2457: inject_eio: local pooltype=rep 2026-02-22T23:22:58.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2458: inject_eio: shift 2026-02-22T23:22:58.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2459: inject_eio: local which=data 2026-02-22T23:22:58.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2460: inject_eio: shift 2026-02-22T23:22:58.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2461: inject_eio: local poolname=test 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2462: inject_eio: shift 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2463: inject_eio: local objname=existing_99 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2464: inject_eio: shift 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2465: inject_eio: local dir=td/divergent-priors 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2466: inject_eio: shift 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:118: TEST_divergent: rados -p test put existing_100 /etc/fstab 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2467: inject_eio: local shard_id=0 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2468: inject_eio: shift 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: get_osds test existing_99 2026-02-22T23:22:58.890 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1021: get_osds: local poolname=test 2026-02-22T23:22:58.891 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1022: get_osds: local objectname=existing_99 2026-02-22T23:22:58.891 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: ceph --format json osd map test existing_99 2026-02-22T23:22:58.891 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: jq '.acting | .[]' 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1025: get_osds: local 'osds=1 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:0 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1027: get_osds: echo 1 0 2 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: initial_osds=('1' '0' '2') 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2470: inject_eio: local -a initial_osds 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2471: inject_eio: local osd_id=1 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2472: inject_eio: '[' rep '!=' ec ']' 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2473: inject_eio: shard_id= 2026-02-22T23:22:59.087 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: cat td/divergent-priors/1/type 2026-02-22T23:22:59.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2475: inject_eio: type=bluestore 2026-02-22T23:22:59.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2476: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-22T23:22:59.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1161: set_config: local daemon=osd 2026-02-22T23:22:59.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1162: set_config: local id=1 2026-02-22T23:22:59.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1163: set_config: local config=bluestore_debug_inject_read_err 2026-02-22T23:22:59.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1164: set_config: local value=true 2026-02-22T23:22:59.089 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: jq 'has("success")' 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: get_asok_path osd.1 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:59.090 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:22:59.091 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.20359/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-22T23:22:59.151 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1168: set_config: test true == true 2026-02-22T23:22:59.151 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2477: inject_eio: local loop=0 2026-02-22T23:22:59.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2479: inject_eio: grep -q Invalid 2026-02-22T23:22:59.152 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: get_asok_path osd.1 2026-02-22T23:22:59.152 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:22:59.152 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:22:59.152 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:22:59.153 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:22:59.153 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:22:59.153 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:22:59.153 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: CEPH_ARGS= 2026-02-22T23:22:59.153 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2478: inject_eio: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok injectdataerr test existing_99 2026-02-22T23:22:59.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:122: TEST_divergent: sleep 10 2026-02-22T23:22:59.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:121: TEST_divergent: rados -p test get existing_99 td/divergent-priors/existing 2026-02-22T23:23:09.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:123: TEST_divergent: killall -9 rados 2026-02-22T23:23:09.259 INFO:tasks.workunit.client.0.trial122.stderr:rados: no process found 2026-02-22T23:23:09.259 INFO:tasks.workunit.client.0.trial122.stdout:killing all the osds 2026-02-22T23:23:09.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:126: TEST_divergent: echo 'killing all the osds' 2026-02-22T23:23:09.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:127: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:09.446 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:09.446 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 72800 0 0 17 84 17 active+clean 2026-02-22T23:22:59.232448+0000 29'101 29:138 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:35:30.942054+0000 0 0 2026-02-22T23:23:09.446 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:09.446 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:09.446 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:09.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:128: TEST_divergent: kill_daemons td/divergent-priors KILL osd 2026-02-22T23:23:09.455 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:23:09.455 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:23:09.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:23:09.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:23:09.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:23:09.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:23:09.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-22T23:23:09.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.0 2026-02-22T23:23:09.816 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:23:09.826 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-22T23:23:09.826 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.1 2026-02-22T23:23:10.077 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:23:10.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:129: TEST_divergent: for i in $osds 2026-02-22T23:23:10.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:131: TEST_divergent: ceph osd down osd.2 2026-02-22T23:23:10.338 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:23:10.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:10.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.0 2026-02-22T23:23:10.611 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already out. 2026-02-22T23:23:10.620 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:133: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:10.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:135: TEST_divergent: ceph osd out osd.2 2026-02-22T23:23:10.914 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already out. 2026-02-22T23:23:10.924 INFO:tasks.workunit.client.0.trial122.stdout:bringing up non_divergent 0 2 2026-02-22T23:23:10.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:139: TEST_divergent: echo 'bringing up non_divergent 0 2' 2026-02-22T23:23:10.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:140: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:11.127 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:11.127 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 72800 0 0 17 84 17 stale+active+clean 2026-02-22T23:22:59.232448+0000 29'101 29:138 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:35:30.942054+0000 0 0 2026-02-22T23:23:11.127 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:11.128 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:11.128 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:11.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 0 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:23:11.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:11.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:23:11.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:23:11.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:23:11.142 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:23:11.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:23:11.143 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-22T23:23:11.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:23:11.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:23:11.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:23:11.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:23:11.163 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:11.161+0000 7fd0002108c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:11.164 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:11.161+0000 7fd0002108c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:11.165 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:11.161+0000 7fd0002108c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:11.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:23:11.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:23:11.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:23:11.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:23:11.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:23:11.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:11.342 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:23:11.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:23:11.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:11.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:23:11.547 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:12.413 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:12.409+0000 7fd0002108c0 -1 Falling back to public interface 2026-02-22T23:23:12.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:12.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:12.548 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:23:12.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:23:12.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:12.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:23:12.748 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:13.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:13.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:13.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:23:13.749 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:23:13.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:13.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:23:13.950 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:14.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:14.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:14.952 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:23:14.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:23:14.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:14.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:23:15.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:15.714 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:15.709+0000 7fd0002108c0 -1 osd.0 29 log_to_monitors true 2026-02-22T23:23:16.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:16.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:16.162 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:23:16.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:23:16.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:16.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:23:16.366 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up out weight 0 up_from 36 up_thru 0 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/364200310,v1:127.0.0.1:6803/364200310] [v2:127.0.0.1:6804/364200310,v1:127.0.0.1:6805/364200310] exists,up 9f76a466-b535-45aa-bf72-3bfe89505d1c 2026-02-22T23:23:16.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:23:16.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:23:16.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:23:16.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:141: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:16.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:143: TEST_divergent: activate_osd td/divergent-priors 2 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:23:16.367 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:23:16.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:23:16.369 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:23:16.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:23:16.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:23:16.369 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-22T23:23:16.370 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:23:16.370 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:23:16.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:23:16.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:23:16.394 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:16.389+0000 7fc4f5fd78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:16.396 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:16.393+0000 7fc4f5fd78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:16.397 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:16.393+0000 7fc4f5fd78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:16.562 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:16.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:23:16.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:17.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:17.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:17.766 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:23:17.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:23:17.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:17.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:23:17.877 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:17.873+0000 7fc4f5fd78c0 -1 Falling back to public interface 2026-02-22T23:23:17.970 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:18.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:18.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:18.971 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:23:18.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:23:18.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:18.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:23:19.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:20.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:20.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:20.177 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:23:20.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:23:20.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:20.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:23:20.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:23:20.981 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:20.977+0000 7fc4f5fd78c0 -1 osd.2 29 log_to_monitors true 2026-02-22T23:23:21.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:23:21.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:21.378 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:23:21.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:23:21.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:21.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:23:21.581 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up out weight 0 up_from 38 up_thru 0 down_at 32 last_clean_interval [13,29) [v2:127.0.0.1:6810/3426863449,v1:127.0.0.1:6811/3426863449] [v2:127.0.0.1:6812/3426863449,v1:127.0.0.1:6813/3426863449] exists,up 6ecf0020-a7e1-44bf-9f24-2a5230b4e516 2026-02-22T23:23:21.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:23:21.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:23:21.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:23:21.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:21.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.0 2026-02-22T23:23:21.827 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already in. 2026-02-22T23:23:21.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:145: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:21.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:147: TEST_divergent: ceph osd in osd.2 2026-02-22T23:23:22.132 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already in. 2026-02-22T23:23:22.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:23:22.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:150: TEST_divergent: wait_for_clean 2026-02-22T23:23:22.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:23:22.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:23:22.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:23:22.143 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:23:22.143 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:23:22.143 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:23:22.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:23:22.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:23:22.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:23:22.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:23:22.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:23:22.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:23:22.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:23:22.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:23:22.209 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:23:22.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:23:22.415 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:23:22.415 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:23:22.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:23:22.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:23:22.415 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:23:22.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822660 2026-02-22T23:23:22.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822660 2026-02-22T23:23:22.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822660' 2026-02-22T23:23:22.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:23:22.494 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:23:22.553 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:23:22.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:23:22.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:23:22.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:23:22.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:23:22.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:23:22.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757251 2026-02-22T23:23:22.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757251 2026-02-22T23:23:22.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-154618822660 2-163208757251' 2026-02-22T23:23:22.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:23:22.637 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-154618822660 2026-02-22T23:23:22.637 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:23:22.638 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:23:22.638 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-154618822660 2026-02-22T23:23:22.639 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:23:22.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822660 2026-02-22T23:23:22.640 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 154618822660 2026-02-22T23:23:22.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 154618822660' 2026-02-22T23:23:22.640 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:23:22.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822658 -lt 154618822660 2026-02-22T23:23:22.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:23:23.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:23:23.839 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:23:24.044 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822660 -lt 154618822660 2026-02-22T23:23:24.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:23:24.045 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-163208757251 2026-02-22T23:23:24.045 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:23:24.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:23:24.047 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-163208757251 2026-02-22T23:23:24.047 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:23:24.048 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757251 2026-02-22T23:23:24.048 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 163208757251' 2026-02-22T23:23:24.049 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 163208757251 2026-02-22T23:23:24.049 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:23:24.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757251 -lt 163208757251 2026-02-22T23:23:24.249 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:23:24.250 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:24.250 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:24.506 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:24.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:24.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:24.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:24.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:24.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:24.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:24.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:23:24.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:23:24.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:23:24.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:24.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:23:25.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:25.054 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:25.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:25.244 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:25.245 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:25.245 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:25.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:25.508 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:23:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:23:25.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:25.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:25.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:25.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:25.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:25.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:25.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:25.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:26.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:26.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:26.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:26.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:26.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:26.453 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:26.453 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:26.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:26.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:26.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:23:26.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:26.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:23:27.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:27.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:27.121 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:27.121 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:27.121 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:27.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:27.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:27.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:27.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:27.326 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:27.326 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:27.326 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:27.596 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:27.597 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:27.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:27.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:27.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:23:27.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:27.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:23:28.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:28.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:28.663 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:28.663 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:28.663 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:28.663 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:28.663 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:28.663 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:28.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:28.858 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:28.859 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:28.859 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:29.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:29.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:29.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:29.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:29.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:29.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:23:29.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:29.369 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:23:30.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:30.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:31.170 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:31.171 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:31.171 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:31.172 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:31.434 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:31.435 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:31.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:31.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:31.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:23:31.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:31.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:23:34.898 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:34.899 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:34.899 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:34.900 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:34.900 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:34.900 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:34.900 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:34.900 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:35.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:35.102 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:35.102 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:35.103 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:35.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:35.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:35.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:35.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:35.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:35.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:35.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:35.357 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:35.357 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:35.617 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:35.617 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:35.617 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:23:35.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:35.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:23:42.017 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:42.017 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:42.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:42.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:42.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:42.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:42.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:42.019 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:42.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:42.213 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:42.213 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:42.213 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:42.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:42.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:42.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:42.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:42.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:42.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:42.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:42.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:23:42.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:23:42.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:23:50.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:23:50.035 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:23:50.035 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:23:50.035 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:23:50.036 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:23:50.036 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:23:50.036 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:23:50.036 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:23:50.229 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:23:50.229 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:23:50.229 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:23:50.229 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:23:50.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:23:50.493 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:23:50.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:23:50.754 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:23:50.755 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:23:50.755 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:23:51.016 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:23:51.016 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "cc2eb9b3-28f4-46d7-8acb-d83e408ece29", 2026-02-22T23:23:51.016 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:23:51.016086+0000", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:23:51.017 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.018 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:23:51.019 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:23:51.020 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.021 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:22:17.385924Z", 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:22:17.385924Z", 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:23:51.022 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:23:51.023 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.024 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:23:51.025 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 41, 2026-02-22T23:23:51.026 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:22:17.464933+0000", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:23:22.182920+0000", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:23:21.136897+0000", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:23:22.080732+0000", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:23:51.027 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 9, 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:23:51.028 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:22:45.085270+0000", 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 1, 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool", 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:23:51.029 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 0, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "off", 2026-02-22T23:23:51.030 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 1, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 1, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 1, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 1, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.031 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "26", 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "0", 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "selfmanaged", 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 0, 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 0, 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [], 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:23:51.032 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:23:51.033 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "", 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 0, 2026-02-22T23:23:51.034 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {}, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "read_balance": { 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "score_type": "Fair distribution", 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "score_acting": 2.9900000095367432, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "score_stable": 2.9900000095367432, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_score": 0.67000001668930054, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_acting": 2, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_stable": 2, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity_weighted": 1, 2026-02-22T23:23:51.035 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity": 1, 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity_weighted": 1 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "9f76a466-b535-45aa-bf72-3bfe89505d1c", 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:23:51.036 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 5, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 29, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 36, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 40, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 30, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.037 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.038 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:23:51.039 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.040 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.041 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 364200310 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/364200310", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/364200310", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/364200310", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/364200310", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:23:51.042 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "fe00adae-737d-4f7c-aec2-9f44e8f9487e", 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 20, 2026-02-22T23:23:51.043 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 31, 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.044 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:23:51.045 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.046 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.047 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3737256882 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3737256882", 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3737256882", 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3737256882", 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3737256882", 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.048 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "6ecf0020-a7e1-44bf-9f24-2a5230b4e516", 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 13, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 29, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 38, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 32, 2026-02-22T23:23:51.049 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.050 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.051 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:23:51.052 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:23:51.053 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3426863449 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3426863449", 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3426863449", 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3426863449", 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3426863449", 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:23:51.054 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:23:09.763783+0000", 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:22:26.165431+0000", 2026-02-22T23:23:51.055 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:23:10.026112+0000", 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:22:35.195070+0000", 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:23:10.286347+0000", 2026-02-22T23:23:51.056 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:22:43.176547+0000", 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:23:51.057 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:23:51.058 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/364200310,v1:127.0.0.1:6805/364200310]", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:23:51.059 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:23:51.060 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:22:19.607834Z", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/364200310,v1:127.0.0.1:6803/364200310]", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/364200310,v1:127.0.0.1:6809/364200310]", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/364200310,v1:127.0.0.1:6807/364200310]", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:23:51.061 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3737256882,v1:127.0.0.1:6813/3737256882]", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:23:51.062 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:23:51.063 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:22:28.815240Z", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3737256882,v1:127.0.0.1:6811/3737256882]", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3737256882,v1:127.0.0.1:6817/3737256882]", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3737256882,v1:127.0.0.1:6815/3737256882]", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:23:51.064 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3426863449,v1:127.0.0.1:6813/3426863449]", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:23:51.065 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:22:36.815714Z", 2026-02-22T23:23:51.066 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3426863449,v1:127.0.0.1:6811/3426863449]", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3426863449,v1:127.0.0.1:6817/3426863449]", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3426863449,v1:127.0.0.1:6815/3426863449]", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:23:51.067 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 26, 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [ 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: "floor": 26 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.068 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 36 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 39 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 41, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:23:51.069 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:23:51.070 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.071 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.072 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.073 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:23:51.074 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.075 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:23:51.076 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.077 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:23:51.078 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stderr:report 238488749 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 0, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:22:17:464715+0000", 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:23:51.079 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:23:51.080 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 1, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 72800, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 100, 2026-02-22T23:23:51.081 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 100, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.082 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 16, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 16, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "up": 2, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 2, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83976, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2112, 2026-02-22T23:23:51.083 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81781, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488824, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036555776, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2162688, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 500149, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83744619 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 1, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:23:51.084 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83976, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2112, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81781, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488824, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036555776, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2162688, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 500149, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:23:51.085 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83744619 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 1, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 72800, 2026-02-22T23:23:51.086 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 100, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 100, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:23:51.087 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1228800, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 218400, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 16, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 16, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "up": 2, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 2, 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.088 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 154618822665 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705677 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 163208757256 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num": 1 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 1, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:23:51.089 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 4, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:23:35.936716+0000", 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "services": { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "osd": { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "daemons": { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "summary": "", 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "0": { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "2": { 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:23:51.090 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 44, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 235, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 331, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: expr 2 + 2 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:153: TEST_divergent: objname=existing_4 2026-02-22T23:23:51.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:154: TEST_divergent: echo 'writing non-divergent object existing_4' 2026-02-22T23:23:51.092 INFO:tasks.workunit.client.0.trial122.stdout:writing non-divergent object existing_4 2026-02-22T23:23:51.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:155: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:51.211 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:51.211 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 29'100 41:136 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:51.211 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:51.211 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:51.211 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:51.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:156: TEST_divergent: rados -p test put existing_4 /etc/resolv.conf 2026-02-22T23:23:51.249 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery 2026-02-22T23:23:51.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:159: TEST_divergent: echo 'delay recovery' 2026-02-22T23:23:51.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:160: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:51.434 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:51.434 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 29'100 41:136 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:51.434 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:51.434 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:51.435 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:51.442 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:51.443 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.0 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2026-02-22T23:23:51.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok set_recovery_delay 100000 2026-02-22T23:23:51.498 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:23:51.503 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:161: TEST_divergent: for i in $non_divergent 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: get_asok_path osd.2 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:23:51.505 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: CEPH_ARGS= 2026-02-22T23:23:51.506 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:163: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok set_recovery_delay 100000 2026-02-22T23:23:51.556 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:23:51.562 INFO:tasks.workunit.client.0.trial122.stdout:revive divergent 1 2026-02-22T23:23:51.562 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:167: TEST_divergent: echo 'revive divergent 1' 2026-02-22T23:23:51.562 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:168: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:51.758 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:51.759 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 29'100 41:136 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:51.759 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:51.759 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:51.759 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:51.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:169: TEST_divergent: ceph osd set noup 2026-02-22T23:23:52.089 INFO:tasks.workunit.client.0.trial122.stderr:noup is set 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:170: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:23:52.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:23:52.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:23:52.102 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:23:52.102 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:23:52.102 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:23:52.102 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:23:52.103 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:23:52.103 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:23:52.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:23:52.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:23:52.119 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:52.117+0000 7ff3a693a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:52.120 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:52.117+0000 7ff3a693a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:52.121 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:52.117+0000 7ff3a693a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:52.306 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-22T23:23:52.306 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:171: TEST_divergent: sleep 5 2026-02-22T23:23:53.349 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:53.345+0000 7ff3a693a8c0 -1 Falling back to public interface 2026-02-22T23:23:56.363 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:56.361+0000 7ff3a693a8c0 -1 osd.1 29 log_to_monitors true 2026-02-22T23:23:57.307 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery divergent 2026-02-22T23:23:57.307 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:173: TEST_divergent: echo 'delay recovery divergent' 2026-02-22T23:23:57.307 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:174: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:57.490 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:57.490 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 41'101 41:137 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:57.490 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:57.490 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:57.490 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: get_asok_path osd.1 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:57.499 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:23:57.500 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: CEPH_ARGS= 2026-02-22T23:23:57.500 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:175: TEST_divergent: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok set_recovery_delay 100000 2026-02-22T23:23:57.553 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:23:57.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:177: TEST_divergent: ceph osd unset noup 2026-02-22T23:23:57.867 INFO:tasks.workunit.client.0.trial122.stderr:noup is unset 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:179: TEST_divergent: wait_for_osd up 0 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:57.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:23:57.878 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:23:57.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:57.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:23:58.080 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 36 up_thru 40 down_at 30 last_clean_interval [5,29) [v2:127.0.0.1:6802/364200310,v1:127.0.0.1:6803/364200310] [v2:127.0.0.1:6804/364200310,v1:127.0.0.1:6805/364200310] exists,up 9f76a466-b535-45aa-bf72-3bfe89505d1c 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:180: TEST_divergent: wait_for_osd up 1 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:58.081 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:23:58.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:23:58.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:58.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:23:58.275 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 31 last_clean_interval [9,29) [v2:127.0.0.1:6818/3939646987,v1:127.0.0.1:6819/3939646987] [v2:127.0.0.1:6820/3939646987,v1:127.0.0.1:6821/3939646987] exists,up fe00adae-737d-4f7c-aec2-9f44e8f9487e 2026-02-22T23:23:58.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:23:58.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:23:58.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:181: TEST_divergent: wait_for_osd up 2 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:23:58.277 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:23:58.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:58.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:23:58.477 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 38 up_thru 0 down_at 32 last_clean_interval [13,29) [v2:127.0.0.1:6810/3426863449,v1:127.0.0.1:6811/3426863449] [v2:127.0.0.1:6812/3426863449,v1:127.0.0.1:6813/3426863449] exists,up 6ecf0020-a7e1-44bf-9f24-2a5230b4e516 2026-02-22T23:23:58.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:23:58.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:23:58.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:23:58.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:183: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:58.668 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:58.668 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 41'101 44:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:58.668 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:58.668 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:58.668 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:58.676 INFO:tasks.workunit.client.0.trial122.stdout:wait for peering 2026-02-22T23:23:58.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:184: TEST_divergent: echo 'wait for peering' 2026-02-22T23:23:58.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:185: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:58.875 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:58.875 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 41'101 44:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:58.875 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:58.875 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:58.875 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:58.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:186: TEST_divergent: rados -p test put foo /etc/fstab 2026-02-22T23:23:58.912 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:188: TEST_divergent: echo 'killing divergent 1' 2026-02-22T23:23:58.912 INFO:tasks.workunit.client.0.trial122.stdout:killing divergent 1 2026-02-22T23:23:58.912 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:189: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:59.096 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:59.096 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 41'101 44:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:59.096 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:59.096 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:59.096 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:59.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:190: TEST_divergent: kill_daemons td/divergent-priors KILL osd.1 2026-02-22T23:23:59.104 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:23:59.104 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:23:59.105 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:23:59.105 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:23:59.105 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:23:59.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:23:59.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:192: TEST_divergent: echo 'reviving divergent 1' 2026-02-22T23:23:59.214 INFO:tasks.workunit.client.0.trial122.stdout:reviving divergent 1 2026-02-22T23:23:59.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:193: TEST_divergent: ceph pg dump pgs 2026-02-22T23:23:59.402 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:23:59.402 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:23:22.187705+0000 41'101 44:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:13:00.448515+0000 0 0 2026-02-22T23:23:59.402 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:23:59.403 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:23:59.403 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:194: TEST_divergent: activate_osd td/divergent-priors 1 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:23:59.412 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:23:59.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:23:59.414 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:23:59.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:23:59.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:23:59.414 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:23:59.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:23:59.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:23:59.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:23:59.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:23:59.441 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:59.437+0000 7f5541dcc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:59.442 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:59.437+0000 7f5541dcc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:59.443 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:23:59.441+0000 7f5541dcc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:23:59.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:23:59.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:23:59.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:23:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:23:59.816 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 46 up_thru 46 down_at 31 last_clean_interval [9,29) [v2:127.0.0.1:6818/3939646987,v1:127.0.0.1:6819/3939646987] [v2:127.0.0.1:6820/3939646987,v1:127.0.0.1:6821/3939646987] exists,up fe00adae-737d-4f7c-aec2-9f44e8f9487e 2026-02-22T23:23:59.817 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:23:59.817 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:23:59.817 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:23:59.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:196: TEST_divergent: sleep 20 2026-02-22T23:24:00.705 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:00.701+0000 7f5541dcc8c0 -1 Falling back to public interface 2026-02-22T23:24:03.685 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:03.681+0000 7f5541dcc8c0 -1 osd.1 47 log_to_monitors true 2026-02-22T23:24:19.818 INFO:tasks.workunit.client.0.trial122.stdout:allowing recovery 2026-02-22T23:24:19.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:198: TEST_divergent: echo 'allowing recovery' 2026-02-22T23:24:19.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:199: TEST_divergent: ceph pg dump pgs 2026-02-22T23:24:20.006 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:24:20.007 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73767 0 0 18 84 18 active+clean 2026-02-22T23:24:04.822715+0000 47'102 51:165 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:07:02.665343+0000 0 0 2026-02-22T23:24:20.007 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:24:20.007 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:24:20.007 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:24:20.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-22T23:24:20.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-22T23:24:20.097 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:24:20.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-22T23:24:20.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-22T23:24:20.180 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:24:20.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:201: TEST_divergent: for i in $osds 2026-02-22T23:24:20.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:203: TEST_divergent: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-22T23:24:20.264 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:24:20.271 INFO:tasks.workunit.client.0.trial122.stdout:reading divergent objects 2026-02-22T23:24:20.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:206: TEST_divergent: echo 'reading divergent objects' 2026-02-22T23:24:20.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:207: TEST_divergent: ceph pg dump pgs 2026-02-22T23:24:20.469 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:24:20.469 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73767 0 0 18 84 18 active+clean 2026-02-22T23:24:04.822715+0000 47'102 51:165 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:22:45.120181+0000 0'0 2026-02-22T23:22:45.120181+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:07:02.665343+0000 0 0 2026-02-22T23:24:20.470 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:24:20.470 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:24:20.470 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:24:20.478 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: expr 2 + 2 2026-02-22T23:24:20.480 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: seq 1 4 2026-02-22T23:24:20.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:24:20.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_1 td/divergent-priors/existing 2026-02-22T23:24:20.502 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:24:20.502 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_2 td/divergent-priors/existing 2026-02-22T23:24:20.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:24:20.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_3 td/divergent-priors/existing 2026-02-22T23:24:20.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:208: TEST_divergent: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:24:20.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:210: TEST_divergent: rados -p test get existing_4 td/divergent-priors/existing 2026-02-22T23:24:20.583 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:212: TEST_divergent: rm -f td/divergent-priors/existing 2026-02-22T23:24:20.585 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-22T23:24:20.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:214: TEST_divergent: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:24:20.590 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4550.0:1 2026-02-22T23:22:58.905526+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 20 _merge_object_divergent_entries: keeping 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4550.0:1 2026-02-22T23:22:58.905526+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 29'100 first_divergent_update: 29'101 last_divergent_update: 29'101 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 29'101 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 29'101 (29'100) modify 1:c156033e:::existing_100:head by client.4550.0:1 2026-02-22T23:22:58.905526+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-22T23:24:20.591 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:23:58.069+0000 7ff3848b8640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-22T23:24:20.592 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: find td/divergent-priors -name '*osd*log' 2026-02-22T23:24:20.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:216: TEST_divergent: grep -q '_merge_object_divergent_entries.*cannot roll back, removing and adding to missing' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:24:20.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:221: TEST_divergent: echo success 2026-02-22T23:24:20.596 INFO:tasks.workunit.client.0.trial122.stdout:success 2026-02-22T23:24:20.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:223: TEST_divergent: delete_pool test 2026-02-22T23:24:20.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-22T23:24:20.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-22T23:24:20.881 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' does not exist 2026-02-22T23:24:20.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:224: TEST_divergent: kill_daemons td/divergent-priors 2026-02-22T23:24:20.891 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:24:20.891 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:24:20.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:24:20.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:24:20.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:24:29.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:24:29.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-22T23:24:29.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:24:29.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:24:29.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:24:29.211 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:24:29.211 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:24:29.212 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:24:29.212 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:24:29.212 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:24:29.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:24:29.217 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:24:29.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:24:29.218 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:24:29.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:24:29.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:24:29.220 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:24:29.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:24:29.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:24:29.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:24:29.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:24:29.223 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:24:29.224 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:24:29.224 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:24:29.241 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:24:29.241 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.241 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:24:29.243 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:24:29.243 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:24:29.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:24:29.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:24:29.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:24:29.246 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:24:29.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:24:29.246 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:24:29.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:24:29.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:24:29.248 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:24:29.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:24:29.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:24:29.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:24:29.250 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:24:29.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:24:29.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:24:29.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:24:29.253 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:24:29.254 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.254 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.254 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:24:29.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:24:29.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:24:29.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-22T23:24:29.256 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:24:29.257 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.257 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20359 2026-02-22T23:24:29.258 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:24:29.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-22T23:24:29.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:24:29.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-22T23:24:29.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_2 td/divergent-priors 2026-02-22T23:24:29.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:421: TEST_divergent_2: local dir=td/divergent-priors 2026-02-22T23:24:29.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:424: TEST_divergent_2: local dummyfile=/etc/fstab 2026-02-22T23:24:29.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:425: TEST_divergent_2: local dummyfile2=/etc/resolv.conf 2026-02-22T23:24:29.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:427: TEST_divergent_2: local num_osds=3 2026-02-22T23:24:29.260 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: expr 3 - 1 2026-02-22T23:24:29.261 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: seq 0 2 2026-02-22T23:24:29.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:428: TEST_divergent_2: local 'osds=0 2026-02-22T23:24:29.262 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:24:29.262 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:24:29.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:429: TEST_divergent_2: run_mon td/divergent-priors a 2026-02-22T23:24:29.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-22T23:24:29.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:24:29.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:24:29.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:24:29.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-22T23:24:29.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-22T23:24:29.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:24:29.285 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:24:29.285 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:24:29.285 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:24:29.285 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.285 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.286 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:29.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:24:29.314 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:24:29.315 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.316 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.317 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:24:29.317 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:24:29.317 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get fsid 2026-02-22T23:24:29.371 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:24:29.372 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:24:29.373 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:24:29.373 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.373 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.373 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:24:29.374 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:24:29.374 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get mon_host 2026-02-22T23:24:29.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:430: TEST_divergent_2: run_mgr td/divergent-priors x 2026-02-22T23:24:29.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-22T23:24:29.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:24:29.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:24:29.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:24:29.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-22T23:24:29.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:24:29.538 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:24:29.538 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:24:29.538 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:24:29.538 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:24:29.538 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.539 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.539 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:29.539 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:24:29.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 0 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:24:29.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:29.562 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:24:29.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:24:29.564 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:24:29.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b016c956-f1df-4da8-af54-22fae7940684 2026-02-22T23:24:29.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 b016c956-f1df-4da8-af54-22fae7940684' 2026-02-22T23:24:29.565 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 b016c956-f1df-4da8-af54-22fae7940684 2026-02-22T23:24:29.565 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:24:29.578 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAtkJtpq695IhAAMEHxuzcbO3cw4i2Vacz/Mg== 2026-02-22T23:24:29.579 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAtkJtpq695IhAAMEHxuzcbO3cw4i2Vacz/Mg=="}' 2026-02-22T23:24:29.579 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b016c956-f1df-4da8-af54-22fae7940684 -i td/divergent-priors/0/new.json 2026-02-22T23:24:29.690 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:24:29.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-22T23:24:29.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAtkJtpq695IhAAMEHxuzcbO3cw4i2Vacz/Mg== --osd-uuid b016c956-f1df-4da8-af54-22fae7940684 2026-02-22T23:24:29.717 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:29.711+0000 7f0c2c0e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:29.718 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:29.711+0000 7f0c2c0e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:29.719 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:29.711+0000 7f0c2c0e08c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:29.720 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:29.715+0000 7f0c2c0e08c0 -1 bdev(0x55edd0117c00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:24:29.720 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:29.715+0000 7f0c2c0e08c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-22T23:24:32.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-22T23:24:32.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:24:32.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:24:32.042 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:24:32.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:24:32.325 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:24:32.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:24:32.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:24:32.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:24:32.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:24:32.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:24:32.344 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:32.339+0000 7fd056e828c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:32.345 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:32.339+0000 7fd056e828c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:32.346 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:32.339+0000 7fd056e828c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:32.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:24:32.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:24:32.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:24:32.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:24:32.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:24:32.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:32.526 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:24:32.526 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:24:32.526 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:32.526 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:24:32.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:33.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:33.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:33.726 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:24:33.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:24:33.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:33.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:24:33.837 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:33.835+0000 7fd056e828c0 -1 Falling back to public interface 2026-02-22T23:24:33.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:34.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:34.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:34.925 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:24:34.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:24:34.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:34.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:24:35.125 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:36.127 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:36.127 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:36.127 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:24:36.127 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:24:36.127 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:36.127 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:24:36.323 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:37.063 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:37.059+0000 7fd056e828c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:24:37.324 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:37.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:37.325 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:24:37.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:24:37.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:37.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:24:37.522 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:38.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:38.524 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:38.524 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:24:38.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:24:38.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:38.525 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:24:38.722 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/402184672,v1:127.0.0.1:6803/402184672] [v2:127.0.0.1:6804/402184672,v1:127.0.0.1:6805/402184672] exists,up b016c956-f1df-4da8-af54-22fae7940684 2026-02-22T23:24:38.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:24:38.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:24:38.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:24:38.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-22T23:24:38.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 1 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:24:38.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:24:38.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:38.731 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:24:38.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:24:38.733 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:24:38.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:24:38.734 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:24:38.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 1a5e6140-34fa-45e2-902a-503c534acc20' 2026-02-22T23:24:38.735 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:24:38.741 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA2kJtpQ30YLBAAnIHrUTbzBelU2KCyTK9ooQ== 2026-02-22T23:24:38.741 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA2kJtpQ30YLBAAnIHrUTbzBelU2KCyTK9ooQ=="}' 2026-02-22T23:24:38.741 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 1a5e6140-34fa-45e2-902a-503c534acc20 -i td/divergent-priors/1/new.json 2026-02-22T23:24:38.941 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:24:38.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-22T23:24:38.950 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA2kJtpQ30YLBAAnIHrUTbzBelU2KCyTK9ooQ== --osd-uuid 1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:24:38.968 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:38.967+0000 7f10023438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:38.970 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:38.967+0000 7f10023438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:38.971 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:38.967+0000 7f10023438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:38.971 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:38.971+0000 7f10023438c0 -1 bdev(0x56380e3f5c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:24:38.971 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:38.971+0000 7f10023438c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-22T23:24:41.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-22T23:24:41.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:24:41.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:24:41.252 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:24:41.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:24:41.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:24:41.520 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:24:41.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:24:41.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:24:41.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:24:41.522 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:24:41.548 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:41.547+0000 7fe0800458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:41.550 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:41.547+0000 7fe0800458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:41.551 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:41.547+0000 7fe0800458c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:41.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:24:41.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:24:41.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:24:41.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:24:41.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:24:41.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:41.723 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:24:41.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:24:41.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:41.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:24:41.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:42.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:42.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:42.924 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:24:42.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:24:42.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:42.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:24:43.045 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:43.043+0000 7fe0800458c0 -1 Falling back to public interface 2026-02-22T23:24:43.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:44.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:44.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:44.121 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:24:44.122 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:24:44.122 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:44.122 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:24:44.316 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:45.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:45.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:45.317 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:24:45.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:24:45.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:45.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:24:45.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:46.294 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:46.291+0000 7fe0800458c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:24:46.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:46.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:46.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:24:46.521 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:24:46.522 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:46.522 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:24:46.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:47.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:47.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:47.734 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:24:47.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:24:47.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:47.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:24:47.929 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3741788012,v1:127.0.0.1:6811/3741788012] [v2:127.0.0.1:6812/3741788012,v1:127.0.0.1:6813/3741788012] exists,up 1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:24:47.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:24:47.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:24:47.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:24:47.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:431: TEST_divergent_2: for i in $osds 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:433: TEST_divergent_2: run_osd td/divergent-priors 2 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:24:47.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:24:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:24:47.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:24:47.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8e68e638-fa21-4cb7-a70d-14feeabb0db8 2026-02-22T23:24:47.936 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 8e68e638-fa21-4cb7-a70d-14feeabb0db8 2026-02-22T23:24:47.937 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 8e68e638-fa21-4cb7-a70d-14feeabb0db8' 2026-02-22T23:24:47.937 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:24:47.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA/kJtpykdzOBAAMaQMfACCEQp1Wxo75dFKBA== 2026-02-22T23:24:47.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA/kJtpykdzOBAAMaQMfACCEQp1Wxo75dFKBA=="}' 2026-02-22T23:24:47.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8e68e638-fa21-4cb7-a70d-14feeabb0db8 -i td/divergent-priors/2/new.json 2026-02-22T23:24:48.145 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:24:48.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-22T23:24:48.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA/kJtpykdzOBAAMaQMfACCEQp1Wxo75dFKBA== --osd-uuid 8e68e638-fa21-4cb7-a70d-14feeabb0db8 2026-02-22T23:24:48.174 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:48.171+0000 7f5fe42338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:48.175 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:48.175+0000 7f5fe42338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:48.176 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:48.175+0000 7f5fe42338c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:48.177 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:48.175+0000 7f5fe42338c0 -1 bdev(0x55ec90571c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:24:48.177 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:48.175+0000 7f5fe42338c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-22T23:24:50.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-22T23:24:50.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:24:50.477 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:24:50.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:24:50.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:24:50.742 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:24:50.742 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:24:50.742 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:24:50.742 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:24:50.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:24:50.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:24:50.771 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:50.767+0000 7fb741c408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:50.772 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:50.771+0000 7fb741c408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:50.773 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:50.771+0000 7fb741c408c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:24:50.951 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:24:50.951 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:24:50.951 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:24:50.951 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:24:50.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:24:50.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:50.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:24:50.952 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:24:50.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:50.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:24:51.148 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:52.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:52.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:52.150 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:24:52.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:24:52.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:52.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:24:52.269 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:52.267+0000 7fb741c408c0 -1 Falling back to public interface 2026-02-22T23:24:52.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:53.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:53.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:53.349 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:24:53.349 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:24:53.349 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:53.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:24:53.542 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:54.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:54.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:54.544 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:24:54.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:24:54.545 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:54.545 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:24:54.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:24:55.287 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:24:55.287+0000 7fb741c408c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:24:55.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:24:55.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:24:55.745 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:24:55.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:24:55.746 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:24:55.746 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:24:55.944 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4177418366,v1:127.0.0.1:6819/4177418366] [v2:127.0.0.1:6820/4177418366,v1:127.0.0.1:6821/4177418366] exists,up 8e68e638-fa21-4cb7-a70d-14feeabb0db8 2026-02-22T23:24:55.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:24:55.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:24:55.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:24:55.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:436: TEST_divergent_2: ceph osd set noout 2026-02-22T23:24:56.241 INFO:tasks.workunit.client.0.trial122.stderr:noout is set 2026-02-22T23:24:56.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:437: TEST_divergent_2: ceph osd set noin 2026-02-22T23:24:56.548 INFO:tasks.workunit.client.0.trial122.stderr:noin is set 2026-02-22T23:24:56.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:438: TEST_divergent_2: ceph osd set nodown 2026-02-22T23:24:56.852 INFO:tasks.workunit.client.0.trial122.stderr:nodown is set 2026-02-22T23:24:56.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:439: TEST_divergent_2: create_pool test 1 1 2026-02-22T23:24:56.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-22T23:24:57.103 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' already exists 2026-02-22T23:24:57.112 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:24:58.113 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:440: TEST_divergent_2: ceph osd pool set test size 3 2026-02-22T23:24:58.414 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 size to 3 2026-02-22T23:24:58.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:441: TEST_divergent_2: ceph osd pool set test min_size 2 2026-02-22T23:24:58.717 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 min_size to 2 2026-02-22T23:24:58.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:443: TEST_divergent_2: flush_pg_stats 2026-02-22T23:24:58.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:24:58.729 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:24:58.930 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:24:58.930 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:24:58.931 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:24:58.931 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:24:58.931 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:24:58.931 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:24:59.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-22T23:24:59.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-22T23:24:59.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-22T23:24:59.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:24:59.011 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:24:59.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-22T23:24:59.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-22T23:24:59.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-22T23:24:59.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:24:59.094 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:24:59.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574851 2026-02-22T23:24:59.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574851 2026-02-22T23:24:59.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-55834574851' 2026-02-22T23:24:59.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:24:59.168 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-22T23:24:59.169 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:24:59.170 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:24:59.171 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-22T23:24:59.171 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:24:59.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-22T23:24:59.172 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836487 2026-02-22T23:24:59.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-22T23:24:59.172 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:24:59.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836487 2026-02-22T23:24:59.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:25:00.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:25:00.380 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:00.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-22T23:25:00.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:00.586 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-22T23:25:00.586 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:00.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:25:00.588 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-22T23:25:00.588 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:00.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-22T23:25:00.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-22T23:25:00.590 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705669 2026-02-22T23:25:00.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:25:00.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-22T23:25:00.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:00.796 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574851 2026-02-22T23:25:00.796 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:00.797 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:25:00.797 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574851 2026-02-22T23:25:00.798 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:00.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574851 2026-02-22T23:25:00.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-22T23:25:00.800 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 55834574851 2026-02-22T23:25:00.800 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:25:01.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-22T23:25:01.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:444: TEST_divergent_2: wait_for_clean 2026-02-22T23:25:01.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:25:01.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:25:01.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:25:01.016 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:25:01.017 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:25:01.017 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:25:01.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:25:01.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:25:01.018 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:25:01.113 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:25:01.113 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:25:01.113 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:25:01.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:25:01.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:25:01.114 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:25:01.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:25:01.312 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:25:01.313 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:25:01.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:25:01.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:01.313 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:25:01.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-22T23:25:01.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-22T23:25:01.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-22T23:25:01.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:01.393 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:25:01.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-22T23:25:01.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-22T23:25:01.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671' 2026-02-22T23:25:01.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:01.473 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:25:01.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574854 2026-02-22T23:25:01.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574854 2026-02-22T23:25:01.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-55834574854' 2026-02-22T23:25:01.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:01.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-22T23:25:01.556 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:01.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:25:01.557 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-22T23:25:01.557 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:01.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-22T23:25:01.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-22T23:25:01.559 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836489 2026-02-22T23:25:01.559 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:01.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-22T23:25:01.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:25:02.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:25:02.769 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:02.970 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-22T23:25:02.970 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:02.971 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-22T23:25:02.971 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:02.972 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:25:02.973 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-22T23:25:02.973 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:02.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-22T23:25:02.974 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705671 2026-02-22T23:25:02.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-22T23:25:02.975 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:25:03.187 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-22T23:25:03.187 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:03.188 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574854 2026-02-22T23:25:03.188 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:03.189 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:25:03.190 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574854 2026-02-22T23:25:03.190 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:03.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574854 2026-02-22T23:25:03.191 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 55834574854 2026-02-22T23:25:03.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574854' 2026-02-22T23:25:03.192 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:25:03.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-22T23:25:03.390 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:25:03.391 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:03.391 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:03.647 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:25:03.647 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:03.647 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:03.647 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:03.648 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:03.648 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:03.648 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:03.648 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:03.835 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:25:03.836 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:03.836 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:03.836 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:04.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:25:04.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:25:04.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:25:04.093 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: ceph pg dump pgs --format=json 2026-02-22T23:25:04.093 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: jq '.pg_stats[0].up_primary' 2026-02-22T23:25:04.278 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:25:04.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:447: TEST_divergent_2: local divergent=1 2026-02-22T23:25:04.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:448: TEST_divergent_2: echo 'primary and soon to be divergent is 1' 2026-02-22T23:25:04.287 INFO:tasks.workunit.client.0.trial122.stdout:primary and soon to be divergent is 1 2026-02-22T23:25:04.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:449: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:25:04.477 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:25:04.477 INFO:tasks.workunit.client.0.trial122.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-22T23:24:57.252488+0000 0'0 26:23 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:05:04.188940+0000 0 0 2026-02-22T23:25:04.477 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:25:04.477 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:25:04.478 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:25:04.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:450: TEST_divergent_2: local non_divergent= 2026-02-22T23:25:04.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 0 = 1 ']' 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0' 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 1 = 1 ']' 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:454: TEST_divergent_2: continue 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:451: TEST_divergent_2: for i in $osds 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:453: TEST_divergent_2: '[' 2 = 1 ']' 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:456: TEST_divergent_2: non_divergent=' 0 2' 2026-02-22T23:25:04.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:459: TEST_divergent_2: echo 'writing initial objects' 2026-02-22T23:25:04.488 INFO:tasks.workunit.client.0.trial122.stdout:writing initial objects 2026-02-22T23:25:04.488 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: seq 1 100 2026-02-22T23:25:04.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_1 /etc/fstab 2026-02-22T23:25:04.508 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.509 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_2 /etc/fstab 2026-02-22T23:25:04.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_3 /etc/fstab 2026-02-22T23:25:04.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_4 /etc/fstab 2026-02-22T23:25:04.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_5 /etc/fstab 2026-02-22T23:25:04.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_6 /etc/fstab 2026-02-22T23:25:04.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_7 /etc/fstab 2026-02-22T23:25:04.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_8 /etc/fstab 2026-02-22T23:25:04.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_9 /etc/fstab 2026-02-22T23:25:04.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_10 /etc/fstab 2026-02-22T23:25:04.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_11 /etc/fstab 2026-02-22T23:25:04.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_12 /etc/fstab 2026-02-22T23:25:04.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_13 /etc/fstab 2026-02-22T23:25:04.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_14 /etc/fstab 2026-02-22T23:25:04.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_15 /etc/fstab 2026-02-22T23:25:04.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_16 /etc/fstab 2026-02-22T23:25:04.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.911 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_17 /etc/fstab 2026-02-22T23:25:04.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_18 /etc/fstab 2026-02-22T23:25:04.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_19 /etc/fstab 2026-02-22T23:25:04.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:04.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_20 /etc/fstab 2026-02-22T23:25:05.021 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.021 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_21 /etc/fstab 2026-02-22T23:25:05.049 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.049 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_22 /etc/fstab 2026-02-22T23:25:05.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_23 /etc/fstab 2026-02-22T23:25:05.105 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.105 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_24 /etc/fstab 2026-02-22T23:25:05.133 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.133 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_25 /etc/fstab 2026-02-22T23:25:05.158 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.158 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_26 /etc/fstab 2026-02-22T23:25:05.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_27 /etc/fstab 2026-02-22T23:25:05.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_28 /etc/fstab 2026-02-22T23:25:05.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_29 /etc/fstab 2026-02-22T23:25:05.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_30 /etc/fstab 2026-02-22T23:25:05.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_31 /etc/fstab 2026-02-22T23:25:05.305 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.305 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_32 /etc/fstab 2026-02-22T23:25:05.333 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.333 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_33 /etc/fstab 2026-02-22T23:25:05.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_34 /etc/fstab 2026-02-22T23:25:05.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_35 /etc/fstab 2026-02-22T23:25:05.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_36 /etc/fstab 2026-02-22T23:25:05.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_37 /etc/fstab 2026-02-22T23:25:05.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_38 /etc/fstab 2026-02-22T23:25:05.501 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.501 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_39 /etc/fstab 2026-02-22T23:25:05.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_40 /etc/fstab 2026-02-22T23:25:05.556 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.556 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_41 /etc/fstab 2026-02-22T23:25:05.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_42 /etc/fstab 2026-02-22T23:25:05.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_43 /etc/fstab 2026-02-22T23:25:05.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_44 /etc/fstab 2026-02-22T23:25:05.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_45 /etc/fstab 2026-02-22T23:25:05.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_46 /etc/fstab 2026-02-22T23:25:05.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_47 /etc/fstab 2026-02-22T23:25:05.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_48 /etc/fstab 2026-02-22T23:25:05.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_49 /etc/fstab 2026-02-22T23:25:05.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_50 /etc/fstab 2026-02-22T23:25:05.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_51 /etc/fstab 2026-02-22T23:25:05.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_52 /etc/fstab 2026-02-22T23:25:05.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_53 /etc/fstab 2026-02-22T23:25:05.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_54 /etc/fstab 2026-02-22T23:25:05.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_55 /etc/fstab 2026-02-22T23:25:05.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_56 /etc/fstab 2026-02-22T23:25:05.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:05.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_57 /etc/fstab 2026-02-22T23:25:06.017 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.017 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_58 /etc/fstab 2026-02-22T23:25:06.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_59 /etc/fstab 2026-02-22T23:25:06.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_60 /etc/fstab 2026-02-22T23:25:06.102 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.102 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_61 /etc/fstab 2026-02-22T23:25:06.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_62 /etc/fstab 2026-02-22T23:25:06.148 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.148 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_63 /etc/fstab 2026-02-22T23:25:06.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_64 /etc/fstab 2026-02-22T23:25:06.185 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.185 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_65 /etc/fstab 2026-02-22T23:25:06.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_66 /etc/fstab 2026-02-22T23:25:06.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_67 /etc/fstab 2026-02-22T23:25:06.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_68 /etc/fstab 2026-02-22T23:25:06.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_69 /etc/fstab 2026-02-22T23:25:06.316 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.316 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_70 /etc/fstab 2026-02-22T23:25:06.344 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.344 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_71 /etc/fstab 2026-02-22T23:25:06.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.373 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_72 /etc/fstab 2026-02-22T23:25:06.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_73 /etc/fstab 2026-02-22T23:25:06.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_74 /etc/fstab 2026-02-22T23:25:06.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_75 /etc/fstab 2026-02-22T23:25:06.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.485 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_76 /etc/fstab 2026-02-22T23:25:06.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_77 /etc/fstab 2026-02-22T23:25:06.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_78 /etc/fstab 2026-02-22T23:25:06.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_79 /etc/fstab 2026-02-22T23:25:06.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_80 /etc/fstab 2026-02-22T23:25:06.616 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.616 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_81 /etc/fstab 2026-02-22T23:25:06.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.638 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_82 /etc/fstab 2026-02-22T23:25:06.665 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.665 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_83 /etc/fstab 2026-02-22T23:25:06.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_84 /etc/fstab 2026-02-22T23:25:06.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_85 /etc/fstab 2026-02-22T23:25:06.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_86 /etc/fstab 2026-02-22T23:25:06.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_87 /etc/fstab 2026-02-22T23:25:06.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_88 /etc/fstab 2026-02-22T23:25:06.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.833 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_89 /etc/fstab 2026-02-22T23:25:06.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_90 /etc/fstab 2026-02-22T23:25:06.888 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.888 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_91 /etc/fstab 2026-02-22T23:25:06.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_92 /etc/fstab 2026-02-22T23:25:06.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_93 /etc/fstab 2026-02-22T23:25:06.972 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:06.973 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_94 /etc/fstab 2026-02-22T23:25:07.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:07.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_95 /etc/fstab 2026-02-22T23:25:07.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:07.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_96 /etc/fstab 2026-02-22T23:25:07.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:07.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_97 /etc/fstab 2026-02-22T23:25:07.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:07.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_98 /etc/fstab 2026-02-22T23:25:07.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:07.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_99 /etc/fstab 2026-02-22T23:25:07.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:461: TEST_divergent_2: for i in $(seq 1 $testobjects) 2026-02-22T23:25:07.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:463: TEST_divergent_2: rados -p test put existing_100 /etc/fstab 2026-02-22T23:25:07.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:25:07.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:466: TEST_divergent_2: wait_for_clean 2026-02-22T23:25:07.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:25:07.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:25:07.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:25:07.161 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:25:07.162 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:25:07.162 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:25:07.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:25:07.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:25:07.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:25:07.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:25:07.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:25:07.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:25:07.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:25:07.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:25:07.233 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:25:07.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:25:07.435 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:25:07.435 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:25:07.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:25:07.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:07.435 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:25:07.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-22T23:25:07.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-22T23:25:07.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-22T23:25:07.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:07.513 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:25:07.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705675 2026-02-22T23:25:07.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705675 2026-02-22T23:25:07.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675' 2026-02-22T23:25:07.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:07.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:25:07.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=55834574857 2026-02-22T23:25:07.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 55834574857 2026-02-22T23:25:07.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574857' 2026-02-22T23:25:07.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:07.671 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-22T23:25:07.671 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:07.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:25:07.671 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-22T23:25:07.672 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:07.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-22T23:25:07.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-22T23:25:07.673 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836492 2026-02-22T23:25:07.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:07.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-22T23:25:07.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:25:08.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:25:08.883 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:09.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-22T23:25:09.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:09.085 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705675 2026-02-22T23:25:09.085 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:09.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:25:09.087 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705675 2026-02-22T23:25:09.087 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:09.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705675 2026-02-22T23:25:09.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-22T23:25:09.089 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705675 2026-02-22T23:25:09.089 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:25:09.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705675 2026-02-22T23:25:09.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:09.290 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-55834574857 2026-02-22T23:25:09.290 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:09.292 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:25:09.292 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-55834574857 2026-02-22T23:25:09.292 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:09.293 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=55834574857 2026-02-22T23:25:09.294 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 55834574857 2026-02-22T23:25:09.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 55834574857' 2026-02-22T23:25:09.294 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:25:09.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-22T23:25:09.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:25:09.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:09.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:09.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:25:09.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:09.772 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:09.772 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:09.772 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:09.772 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:09.773 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:09.773 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:09.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:25:09.970 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:09.970 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:09.970 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:10.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:25:10.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:25:10.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:25:10.237 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: get_pg test existing_1 2026-02-22T23:25:10.237 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-22T23:25:10.237 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-22T23:25:10.237 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-22T23:25:10.237 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-22T23:25:10.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:468: TEST_divergent_2: local pgid=1.0 2026-02-22T23:25:10.437 INFO:tasks.workunit.client.0.trial122.stdout:blackholing osds 0 2 2026-02-22T23:25:10.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:471: TEST_divergent_2: echo 'blackholing osds 0 2' 2026-02-22T23:25:10.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:472: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:25:10.624 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:25:10.624 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-22T23:24:57.252488+0000 26'100 26:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:05:04.188940+0000 0 0 2026-02-22T23:25:10.624 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:25:10.624 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:25:10.624 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:25:10.632 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:10.633 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.0 2026-02-22T23:25:10.633 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:25:10.633 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:25:10.633 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:25:10.633 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:25:10.633 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:25:10.634 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:25:10.634 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2026-02-22T23:25:10.634 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-22T23:25:10.685 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:25:10.685 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:25:10.685 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:25:10.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:473: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:10.691 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: get_asok_path osd.2 2026-02-22T23:25:10.691 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:25:10.691 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:25:10.691 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:25:10.691 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:25:10.691 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:25:10.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:25:10.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: CEPH_ARGS= 2026-02-22T23:25:10.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:475: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-22T23:25:10.743 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:25:10.743 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:25:10.743 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:25:10.748 INFO:tasks.workunit.client.0.trial122.stdout:create new divergent objects 2026-02-22T23:25:10.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:479: TEST_divergent_2: echo 'create new divergent objects' 2026-02-22T23:25:10.749 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: seq 1 2 2026-02-22T23:25:10.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:25:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:480: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:25:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_1 2026-02-22T23:25:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:485: TEST_divergent_2: echo 'writing divergent objects' 2026-02-22T23:25:10.750 INFO:tasks.workunit.client.0.trial122.stdout:writing divergent objects 2026-02-22T23:25:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:482: TEST_divergent_2: rados -p test create newobject_2 2026-02-22T23:25:10.751 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: seq 1 2 2026-02-22T23:25:10.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-22T23:25:10.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:486: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-22T23:25:10.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_1 /etc/resolv.conf 2026-02-22T23:25:10.752 INFO:tasks.workunit.client.0.trial122.stdout:remove divergent objects 2026-02-22T23:25:10.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:491: TEST_divergent_2: echo 'remove divergent objects' 2026-02-22T23:25:10.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:488: TEST_divergent_2: rados -p test put existing_2 /etc/resolv.conf 2026-02-22T23:25:10.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: seq 1 2 2026-02-22T23:25:10.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-22T23:25:10.754 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 1 + 2 2026-02-22T23:25:10.755 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=3 2026-02-22T23:25:10.755 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:492: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-22T23:25:10.756 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_3 2026-02-22T23:25:10.756 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: expr 2 + 2 2026-02-22T23:25:10.757 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:494: TEST_divergent_2: rmi=4 2026-02-22T23:25:10.757 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:497: TEST_divergent_2: sleep 10 2026-02-22T23:25:10.757 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:495: TEST_divergent_2: rados -p test rm existing_4 2026-02-22T23:25:20.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:498: TEST_divergent_2: killall -9 rados 2026-02-22T23:25:20.808 INFO:tasks.workunit.client.0.trial122.stderr:rados: no process found 2026-02-22T23:25:20.808 INFO:tasks.workunit.client.0.trial122.stdout:killing all the osds 2026-02-22T23:25:20.809 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:501: TEST_divergent_2: echo 'killing all the osds' 2026-02-22T23:25:20.809 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:502: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:25:20.992 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:25:20.992 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 active+clean 2026-02-22T23:24:57.252488+0000 26'106 26:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:05:04.188940+0000 0 0 2026-02-22T23:25:20.992 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:25:20.993 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:25:20.993 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:25:21.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:503: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd 2026-02-22T23:25:21.001 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:25:21.001 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:25:21.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:25:21.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:25:21.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:25:21.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:25:21.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-22T23:25:21.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.0 2026-02-22T23:25:21.352 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:25:21.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-22T23:25:21.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.1 2026-02-22T23:25:21.620 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:25:21.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:504: TEST_divergent_2: for i in $osds 2026-02-22T23:25:21.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:506: TEST_divergent_2: ceph osd down osd.2 2026-02-22T23:25:21.890 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:25:21.899 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:21.899 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.0 2026-02-22T23:25:22.153 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already out. 2026-02-22T23:25:22.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:508: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:22.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:510: TEST_divergent_2: ceph osd out osd.2 2026-02-22T23:25:22.425 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already out. 2026-02-22T23:25:22.434 INFO:tasks.workunit.client.0.trial122.stdout:bringing up non_divergent 0 2 2026-02-22T23:25:22.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:514: TEST_divergent_2: echo 'bringing up non_divergent 0 2' 2026-02-22T23:25:22.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:515: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:25:22.632 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:25:22.632 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 stale+active+clean 2026-02-22T23:24:57.252488+0000 26'106 26:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:05:04.188940+0000 0 0 2026-02-22T23:25:22.632 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:25:22.632 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:25:22.632 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:25:22.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:22.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 0 2026-02-22T23:25:22.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:25:22.642 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:25:22.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:25:22.644 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:25:22.644 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:25:22.644 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:25:22.645 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-22T23:25:22.646 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:25:22.646 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:25:22.646 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:25:22.646 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:25:22.670 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:22.666+0000 7f8d28d2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:25:22.672 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:22.670+0000 7f8d28d2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:25:22.673 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:22.670+0000 7f8d28d2e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:25:22.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:25:22.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:25:22.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:25:22.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:25:22.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:25:22.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:22.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:25:22.863 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:25:22.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:22.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:25:23.061 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:23.982 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:23.978+0000 7f8d28d2e8c0 -1 Falling back to public interface 2026-02-22T23:25:24.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:24.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:24.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:25:24.063 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:25:24.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:24.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:25:24.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:25.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:25.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:25.262 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:25:25.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:25:25.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:25.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:25:25.456 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:26.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:26.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:26.457 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:25:26.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:25:26.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:26.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:25:26.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:27.071 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:27.066+0000 7f8d28d2e8c0 -1 osd.0 26 log_to_monitors true 2026-02-22T23:25:27.652 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:27.652 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:27.652 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:25:27.653 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:25:27.653 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:27.653 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up out weight 0 up_from 33 up_thru 0 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6802/3009282680,v1:127.0.0.1:6803/3009282680] [v2:127.0.0.1:6804/3009282680,v1:127.0.0.1:6805/3009282680] exists,up b016c956-f1df-4da8-af54-22fae7940684 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:516: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:518: TEST_divergent_2: activate_osd td/divergent-priors 2 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:25:27.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:25:27.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:25:27.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:25:27.852 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:25:27.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:25:27.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:25:27.852 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-22T23:25:27.853 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:25:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:25:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:25:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:25:27.868 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:27.866+0000 7f6b99afd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:25:27.870 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:27.866+0000 7f6b99afd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:25:27.871 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:27.866+0000 7f6b99afd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:25:28.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:25:28.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:25:28.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:25:28.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:25:28.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:25:28.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:28.059 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:25:28.059 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:25:28.059 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:28.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:25:28.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:29.121 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:29.118+0000 7f6b99afd8c0 -1 Falling back to public interface 2026-02-22T23:25:29.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:29.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:29.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:25:29.257 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:25:29.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:29.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:25:29.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:30.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:30.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:30.454 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:25:30.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:25:30.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:30.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:25:30.655 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:31.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:31.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:31.657 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:25:31.658 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:25:31.658 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:31.659 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:25:31.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:25:32.406 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:25:32.401+0000 7f6b99afd8c0 -1 osd.2 26 log_to_monitors true 2026-02-22T23:25:32.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:25:32.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:25:32.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:25:32.858 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:25:32.859 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:25:32.859 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:25:33.056 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up out weight 0 up_from 36 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/95227242,v1:127.0.0.1:6811/95227242] [v2:127.0.0.1:6812/95227242,v1:127.0.0.1:6813/95227242] exists,up 8e68e638-fa21-4cb7-a70d-14feeabb0db8 2026-02-22T23:25:33.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:25:33.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:25:33.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:25:33.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:33.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.0 2026-02-22T23:25:33.302 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already in. 2026-02-22T23:25:33.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:520: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:25:33.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:522: TEST_divergent_2: ceph osd in osd.2 2026-02-22T23:25:33.607 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already in. 2026-02-22T23:25:33.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:25:33.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:525: TEST_divergent_2: wait_for_clean 2026-02-22T23:25:33.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:25:33.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:25:33.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:25:33.619 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:25:33.619 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:25:33.620 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:25:33.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:25:33.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:25:33.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:25:33.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:25:33.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:25:33.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:25:33.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:25:33.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:25:33.677 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:25:33.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:25:33.881 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:25:33.881 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:25:33.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:25:33.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:33.881 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:25:33.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920772 2026-02-22T23:25:33.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920772 2026-02-22T23:25:33.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920772' 2026-02-22T23:25:33.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:33.964 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:25:34.022 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:25:34.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:25:34.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:25:34.025 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:25:34.025 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:25:34.025 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:25:34.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822659 2026-02-22T23:25:34.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822659 2026-02-22T23:25:34.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920772 2-154618822659' 2026-02-22T23:25:34.104 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:34.105 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-141733920772 2026-02-22T23:25:34.105 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:34.106 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:25:34.107 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-141733920772 2026-02-22T23:25:34.107 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:34.108 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920772 2026-02-22T23:25:34.109 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 141733920772' 2026-02-22T23:25:34.109 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 141733920772 2026-02-22T23:25:34.109 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:34.305 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920772 2026-02-22T23:25:34.305 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:25:35.306 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:25:35.307 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:35.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920770 -lt 141733920772 2026-02-22T23:25:35.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:25:36.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-22T23:25:36.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:25:36.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920772 -lt 141733920772 2026-02-22T23:25:36.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:25:36.715 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-154618822659 2026-02-22T23:25:36.715 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:25:36.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:25:36.716 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-154618822659 2026-02-22T23:25:36.717 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:25:36.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822659 2026-02-22T23:25:36.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 154618822659' 2026-02-22T23:25:36.718 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 154618822659 2026-02-22T23:25:36.719 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:25:36.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822659 -lt 154618822659 2026-02-22T23:25:36.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:25:36.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:36.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:37.189 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:37.384 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:37.385 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:37.385 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:37.385 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:37.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:37.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:25:37.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:25:37.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:25:37.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:37.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:25:37.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:37.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:37.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:37.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:37.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:37.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:37.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:37.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:37.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:37.952 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:37.952 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:37.952 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:38.218 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:38.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:38.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:38.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:25:38.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:38.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:25:38.687 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:38.687 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:38.687 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:38.687 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:38.687 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:38.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:38.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:38.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:38.884 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:38.885 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:38.885 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:38.885 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:39.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:39.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:39.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:39.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:39.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:39.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:39.153 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:39.153 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:39.153 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:39.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:39.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:39.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:25:39.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:39.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:25:39.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:39.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:39.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:39.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:39.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:39.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:39.821 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:39.821 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:40.021 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:40.021 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:40.022 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:40.022 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:40.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:40.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:40.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:40.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:40.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:40.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:40.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:40.290 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:40.290 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:40.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:40.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:40.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:25:40.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:40.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:25:41.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:41.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:41.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:41.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:41.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:41.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:41.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:41.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:41.552 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:41.553 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:41.553 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:41.553 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:41.808 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:42.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:42.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:42.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:25:42.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:42.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:25:43.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:43.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:43.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:43.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:43.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:43.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:43.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:43.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:43.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:43.865 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:43.865 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:43.865 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:44.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:44.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:44.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:44.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:25:44.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:44.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:25:47.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:47.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:47.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:47.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:47.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:47.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:47.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:47.578 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:47.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:47.771 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:47.771 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:47.771 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:48.031 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:48.032 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:48.293 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:48.293 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:48.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:25:48.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:48.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:25:54.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:25:54.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:25:54.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:25:54.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:25:54.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:25:54.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:25:54.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:25:54.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:25:54.897 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:25:54.898 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:25:54.898 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:25:54.898 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:25:55.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:25:55.165 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:25:55.165 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:25:55.431 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:25:55.431 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:25:55.431 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:25:55.432 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:25:55.432 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:26:02.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:02.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:02.733 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:02.733 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:02.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:02.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:02.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:02.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:02.931 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:02.932 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:02.932 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:02.932 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:03.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:03.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:03.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:03.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:03.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:03.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:03.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:03.199 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:03.199 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:03.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:03.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:03.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:26:03.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "e712ed6b-05a6-4a72-b530-5a83e9f33072", 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:26:03.709512+0000", 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:26:03.711 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.712 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.713 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded", 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.714 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:26:03.715 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:26:03.716 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:24:29.277283Z", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:24:29.277283Z", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:26:03.717 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.718 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.719 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.720 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 40, 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:24:29.356923+0000", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:25:33.657965+0000", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:25:32.562008+0000", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:25:33.555308+0000", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:26:03.721 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 9, 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:26:03.722 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:24:57.017646+0000", 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 1, 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool", 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:26:03.723 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 0, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "off", 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 1, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:26:03.724 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 1, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 1, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 1, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "26", 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:26:03.725 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "0", 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "selfmanaged", 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 0, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 0, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [], 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:26:03.726 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "", 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.727 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {}, 2026-02-22T23:26:03.728 INFO:tasks.workunit.client.0.trial122.stdout: "read_balance": { 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "score_type": "Fair distribution", 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "score_acting": 2.9900000095367432, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "score_stable": 2.9900000095367432, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_score": 0.67000001668930054, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_acting": 2, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_stable": 2, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity_weighted": 1, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity": 1, 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity_weighted": 1 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.729 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "b016c956-f1df-4da8-af54-22fae7940684", 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 5, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 26, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 33, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 39, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 27, 2026-02-22T23:26:03.730 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.731 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.732 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.733 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.734 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/3009282680", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/3009282680", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3009282680", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3009282680", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.735 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "1a5e6140-34fa-45e2-902a-503c534acc20", 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 20, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 28, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.736 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.737 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.738 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:26:03.739 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3741788012", 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3741788012", 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3741788012", 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3741788012", 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.740 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "8e68e638-fa21-4cb7-a70d-14feeabb0db8", 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 13, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 26, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 36, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 29, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.741 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.742 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:26:03.743 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.744 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/95227242", 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/95227242", 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/95227242", 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/95227242", 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:25:21.302626+0000", 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:26:03.745 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:24:38.104758+0000", 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:25:21.567525+0000", 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:25:21.838258+0000", 2026-02-22T23:26:03.746 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:24:56.323082+0000", 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:26:03.747 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.748 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/3009282680,v1:127.0.0.1:6805/3009282680]", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:26:03.749 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:24:31.504372Z", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/3009282680,v1:127.0.0.1:6803/3009282680]", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3009282680,v1:127.0.0.1:6809/3009282680]", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3009282680,v1:127.0.0.1:6807/3009282680]", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:26:03.750 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3741788012,v1:127.0.0.1:6813/3741788012]", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:26:03.751 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:24:40.716607Z", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:26:03.752 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3741788012,v1:127.0.0.1:6811/3741788012]", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3741788012,v1:127.0.0.1:6817/3741788012]", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3741788012,v1:127.0.0.1:6815/3741788012]", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.753 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/95227242,v1:127.0.0.1:6813/95227242]", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:26:03.754 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:24:49.927822Z", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/95227242,v1:127.0.0.1:6811/95227242]", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/95227242,v1:127.0.0.1:6817/95227242]", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/95227242,v1:127.0.0.1:6815/95227242]", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:26:03.755 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 0, 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [] 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 33 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 37 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.756 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 40, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.757 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:26:03.758 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:26:03.759 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:03.760 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.761 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.762 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:26:03.763 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 0, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:26:03.764 INFO:tasks.workunit.client.0.trial122.stderr:report 2252268489 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:24:29:356714+0000", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:26:03.765 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 1, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 72800, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:26:03.766 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 100, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 100, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:26:03.767 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 16, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 16, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "up": 2, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 2, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:26:03.768 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83880, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2064, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488920, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036654080, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2113536, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 485382, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10383, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679089 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:26:03.769 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83880, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2064, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488920, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036654080, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2113536, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 485382, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10383, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679089 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:26:03.770 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 72800, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 100, 2026-02-22T23:26:03.771 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 100, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1220608, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 217422, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:03.772 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 16, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 16, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "up": 2, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 2, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 141733920777 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705677 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 154618822664 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "num": 1 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.773 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 1, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 4, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:25:47.844033+0000", 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "services": { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "osd": { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "daemons": { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "summary": "", 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "0": { 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:26:03.774 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "2": { 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 44, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 232, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 326, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:26:03.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:26:03.776 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: expr 2 + 2 2026-02-22T23:26:03.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:528: TEST_divergent_2: objname=existing_4 2026-02-22T23:26:03.776 INFO:tasks.workunit.client.0.trial122.stdout:writing non-divergent object existing_4 2026-02-22T23:26:03.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:529: TEST_divergent_2: echo 'writing non-divergent object existing_4' 2026-02-22T23:26:03.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:530: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:03.907 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:03.908 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 72800 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 26'100 40:131 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:03.908 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:03.908 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:03.908 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:03.918 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:531: TEST_divergent_2: rados -p test put existing_4 /etc/resolv.conf 2026-02-22T23:26:03.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:26:03.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:533: TEST_divergent_2: wait_for_clean 2026-02-22T23:26:03.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:26:03.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:26:03.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:26:03.942 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:26:03.942 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:26:03.942 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:26:03.942 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:26:03.942 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:26:03.943 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:26:04.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:26:04.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:26:04.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:26:04.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:26:04.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:26:04.003 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:26:04.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:26:04.199 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:26:04.199 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:26:04.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:26:04.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:26:04.199 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:26:04.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920780 2026-02-22T23:26:04.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920780 2026-02-22T23:26:04.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920780' 2026-02-22T23:26:04.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:26:04.279 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:26:04.337 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:26:04.340 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:26:04.340 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:26:04.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:26:04.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:26:04.341 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:26:04.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822667 2026-02-22T23:26:04.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822667 2026-02-22T23:26:04.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920780 2-154618822667' 2026-02-22T23:26:04.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:26:04.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-141733920780 2026-02-22T23:26:04.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:26:04.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:26:04.420 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-141733920780 2026-02-22T23:26:04.421 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:26:04.422 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920780 2026-02-22T23:26:04.422 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 141733920780' 2026-02-22T23:26:04.422 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 141733920780 2026-02-22T23:26:04.422 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:26:04.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920778 -lt 141733920780 2026-02-22T23:26:04.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:26:05.636 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:26:05.637 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:26:05.835 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920778 -lt 141733920780 2026-02-22T23:26:05.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:26:06.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-22T23:26:06.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:26:07.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920780 -lt 141733920780 2026-02-22T23:26:07.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:26:07.040 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-154618822667 2026-02-22T23:26:07.040 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:26:07.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:26:07.042 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-154618822667 2026-02-22T23:26:07.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:26:07.044 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822667 2026-02-22T23:26:07.044 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 154618822667 2026-02-22T23:26:07.044 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 154618822667' 2026-02-22T23:26:07.044 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:26:07.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822667 -lt 154618822667 2026-02-22T23:26:07.253 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:26:07.253 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:07.254 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:07.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:26:07.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:07.511 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:07.511 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:07.511 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:07.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:07.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:07.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:07.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:07.710 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:07.710 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:07.711 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:07.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:07.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:26:07.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:26:07.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:26:07.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:07.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:26:08.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:08.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:08.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:08.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:08.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:08.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:08.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:08.080 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:08.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:08.280 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:08.280 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:08.280 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:08.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:08.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:08.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:08.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:26:08.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:08.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:26:09.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:09.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:09.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:09.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:09.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:09.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:09.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:09.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:09.202 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:09.203 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:09.203 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:09.203 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:09.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:09.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:09.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:09.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:09.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:09.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:09.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:09.464 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:09.464 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:09.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:09.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:09.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:26:09.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:09.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:26:10.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:10.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:10.121 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:10.121 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:10.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:10.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:10.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:10.122 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:10.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:10.318 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:10.318 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:10.318 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:10.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:10.575 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:10.575 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:10.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:10.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:10.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:26:10.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:10.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:26:11.638 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:11.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:11.639 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:11.640 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:11.640 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:11.640 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:11.640 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:11.640 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:11.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:11.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:11.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:11.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:12.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:12.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:12.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:12.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:12.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:12.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:12.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:12.098 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:12.098 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:12.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:12.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:12.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:26:12.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:12.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:26:13.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:13.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:13.955 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:13.955 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:13.955 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:13.955 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:13.955 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:13.955 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:14.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:14.150 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:14.150 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:14.150 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:14.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:14.409 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:14.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:14.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:14.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:26:14.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:14.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:26:17.869 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:17.869 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:17.870 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:17.870 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:17.870 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:17.871 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:17.871 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:17.871 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:18.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:18.068 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:18.069 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:18.069 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:18.330 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:18.331 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:18.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:18.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:18.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:26:18.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:18.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:26:24.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:24.985 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:24.986 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:24.986 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:24.986 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:24.986 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:24.986 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:24.986 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:25.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:25.181 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:25.181 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:25.182 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:25.440 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:25.440 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:25.440 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:25.440 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:25.440 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:25.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:25.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:25.441 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:25.441 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:25.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:25.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:25.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:26:25.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:26:25.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:26:32.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:26:32.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:26:32.997 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:26:32.997 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:26:32.997 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:26:32.997 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:26:32.998 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:26:32.998 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:26:33.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:26:33.193 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:26:33.193 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:26:33.193 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:26:33.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:26:33.450 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:26:33.450 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:26:33.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:26:33.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:26:33.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:26:33.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:26:33.968 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:26:33.968 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "e712ed6b-05a6-4a72-b530-5a83e9f33072", 2026-02-22T23:26:33.968 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:33.968 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:26:33.968 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:26:33.966977+0000", 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.969 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:26:33.970 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 1 pg degraded, 1 pg undersized", 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "count": 2 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.971 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is stuck undersized for 60s, current state active+undersized+degraded, last acting [0,2]" 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:26:33.972 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:26:33.973 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:24:29.277283Z", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:24:29.277283Z", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:26:33.974 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:26:33.975 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.976 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.977 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 40, 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:24:29.356923+0000", 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:25:33.657965+0000", 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:25:32.562008+0000", 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:25:33.555308+0000", 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:26:33.978 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 9, 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:26:33.979 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:26:33.980 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:24:57.017646+0000", 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 1, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool", 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 0, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:26:33.981 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "off", 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 1, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 1, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 1, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 1, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:26:33.982 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "26", 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "0", 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "selfmanaged", 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 0, 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 0, 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [], 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:26:33.983 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "", 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:26:33.984 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:26:33.985 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {}, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "read_balance": { 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "score_type": "Fair distribution", 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "score_acting": 2.9900000095367432, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "score_stable": 2.9900000095367432, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_score": 0.67000001668930054, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_acting": 2, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_stable": 2, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity_weighted": 1, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity": 1, 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity_weighted": 1 2026-02-22T23:26:33.986 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "b016c956-f1df-4da8-af54-22fae7940684", 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 5, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 26, 2026-02-22T23:26:33.987 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 33, 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 39, 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 27, 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.988 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.989 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.990 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:26:33.991 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3009282680 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/3009282680", 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/3009282680", 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/3009282680", 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/3009282680", 2026-02-22T23:26:33.992 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "1a5e6140-34fa-45e2-902a-503c534acc20", 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:26:33.993 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 20, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 28, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.994 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.995 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:26:33.996 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.997 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3741788012 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3741788012", 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3741788012", 2026-02-22T23:26:33.998 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3741788012", 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3741788012", 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "8e68e638-fa21-4cb7-a70d-14feeabb0db8", 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:26:33.999 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 13, 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 26, 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 36, 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 29, 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.000 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:26:34.001 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.002 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.003 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.004 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 95227242 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.005 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/95227242", 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/95227242", 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/95227242", 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/95227242", 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:26:34.006 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:25:21.302626+0000", 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:24:38.104758+0000", 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:25:21.567525+0000", 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:26:34.007 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "0.000000", 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:25:21.838258+0000", 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:24:56.323082+0000", 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.008 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.009 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:26:34.010 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/3009282680,v1:127.0.0.1:6805/3009282680]", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:26:34.011 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:24:31.504372Z", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/3009282680,v1:127.0.0.1:6803/3009282680]", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:26:34.012 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/3009282680,v1:127.0.0.1:6809/3009282680]", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/3009282680,v1:127.0.0.1:6807/3009282680]", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.013 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3741788012,v1:127.0.0.1:6813/3741788012]", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:26:34.014 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:24:40.716607Z", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3741788012,v1:127.0.0.1:6811/3741788012]", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:26:34.015 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3741788012,v1:127.0.0.1:6817/3741788012]", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3741788012,v1:127.0.0.1:6815/3741788012]", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.016 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/95227242,v1:127.0.0.1:6813/95227242]", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:26:34.017 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:24:49.927822Z", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/95227242,v1:127.0.0.1:6811/95227242]", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:26:34.018 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/95227242,v1:127.0.0.1:6817/95227242]", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/95227242,v1:127.0.0.1:6815/95227242]", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:26:34.019 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 0, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [] 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 33 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 37 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 40, 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:26:34.020 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.021 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:26:34.022 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.023 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:26:34.024 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:34.025 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:26:34.026 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:26:34.027 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:26:34.028 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 0, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stderr:report 2287489427 2026-02-22T23:26:34.029 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:24:29:356714+0000", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:26:34.030 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 1, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73039, 2026-02-22T23:26:34.031 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:26:34.032 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 17, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 17, 2026-02-22T23:26:34.033 INFO:tasks.workunit.client.0.trial122.stdout: "up": 2, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 2, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83880, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2064, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488920, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036654080, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2113536, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 485860, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10383, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679089 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.034 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83880, 2026-02-22T23:26:34.035 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2064, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488920, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036654080, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2113536, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 485860, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10383, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679089 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.036 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73039, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:26:34.037 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1220608, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 217900, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:26:34.038 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 17, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 17, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "up": 2, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 2, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 141733920786 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705677 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 154618822673 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "num": 1 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.039 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 6, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:26:33.848897+0000", 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "services": {} 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 60, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:26:34.040 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 286, 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 396, 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:536: TEST_divergent_2: echo 'delay recovery' 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery 2026-02-22T23:26:34.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:537: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:34.165 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:34.165 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 40:132 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:34.165 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:34.165 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:34.165 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:34.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:26:34.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.0 2026-02-22T23:26:34.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:26:34.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:26:34.174 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:26:34.174 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:26:34.174 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:26:34.174 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:26:34.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2026-02-22T23:26:34.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok set_recovery_delay 100000 2026-02-22T23:26:34.227 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:26:34.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:538: TEST_divergent_2: for i in $non_divergent 2026-02-22T23:26:34.233 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: get_asok_path osd.2 2026-02-22T23:26:34.234 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:26:34.234 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:26:34.235 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:26:34.235 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:26:34.235 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:26:34.235 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:26:34.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: CEPH_ARGS= 2026-02-22T23:26:34.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:540: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok set_recovery_delay 100000 2026-02-22T23:26:34.286 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:26:34.291 INFO:tasks.workunit.client.0.trial122.stdout:revive divergent 1 2026-02-22T23:26:34.291 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:544: TEST_divergent_2: echo 'revive divergent 1' 2026-02-22T23:26:34.291 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:545: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:34.488 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:34.488 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 40:132 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:34.488 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:34.488 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:34.489 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:34.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:546: TEST_divergent_2: ceph osd set noup 2026-02-22T23:26:34.828 INFO:tasks.workunit.client.0.trial122.stderr:noup is set 2026-02-22T23:26:34.841 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:547: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:26:34.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:26:34.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:26:34.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:26:34.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:26:34.845 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:26:34.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:26:34.846 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:26:34.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:26:34.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:26:34.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:26:34.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:26:34.870 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:34.864+0000 7f6af5aff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:26:34.871 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:34.864+0000 7f6af5aff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:26:34.872 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:34.868+0000 7f6af5aff8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:26:35.039 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-22T23:26:35.039 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:548: TEST_divergent_2: sleep 5 2026-02-22T23:26:36.105 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:36.100+0000 7f6af5aff8c0 -1 Falling back to public interface 2026-02-22T23:26:39.118 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:39.112+0000 7f6af5aff8c0 -1 osd.1 26 log_to_monitors true 2026-02-22T23:26:40.040 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery divergent 2026-02-22T23:26:40.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:550: TEST_divergent_2: echo 'delay recovery divergent' 2026-02-22T23:26:40.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:551: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:40.227 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:40.227 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 42:135 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:40.227 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:40.227 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:40.227 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:40.235 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: get_asok_path osd.1 2026-02-22T23:26:40.235 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:26:40.235 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:26:40.236 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:26:40.236 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:26:40.236 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:26:40.236 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:26:40.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: CEPH_ARGS= 2026-02-22T23:26:40.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:552: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok set_recovery_delay 100000 2026-02-22T23:26:40.290 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:26:40.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:554: TEST_divergent_2: ceph osd unset noup 2026-02-22T23:26:40.608 INFO:tasks.workunit.client.0.trial122.stderr:noup is unset 2026-02-22T23:26:40.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:556: TEST_divergent_2: wait_for_osd up 0 2026-02-22T23:26:40.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:26:40.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:26:40.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:26:40.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:26:40.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:26:40.619 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:26:40.620 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:26:40.620 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:26:40.620 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:26:40.822 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 33 up_thru 39 down_at 27 last_clean_interval [5,26) [v2:127.0.0.1:6802/3009282680,v1:127.0.0.1:6803/3009282680] [v2:127.0.0.1:6804/3009282680,v1:127.0.0.1:6805/3009282680] exists,up b016c956-f1df-4da8-af54-22fae7940684 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:557: TEST_divergent_2: wait_for_osd up 1 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:26:40.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:26:40.825 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:26:40.825 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:26:40.825 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:26:41.021 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/360547886,v1:127.0.0.1:6819/360547886] [v2:127.0.0.1:6820/360547886,v1:127.0.0.1:6821/360547886] exists,up 1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:558: TEST_divergent_2: wait_for_osd up 2 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:26:41.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:26:41.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:26:41.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:26:41.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:26:41.023 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:26:41.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:26:41.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:26:41.229 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 36 up_thru 0 down_at 29 last_clean_interval [13,26) [v2:127.0.0.1:6810/95227242,v1:127.0.0.1:6811/95227242] [v2:127.0.0.1:6812/95227242,v1:127.0.0.1:6813/95227242] exists,up 8e68e638-fa21-4cb7-a70d-14feeabb0db8 2026-02-22T23:26:41.230 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:26:41.230 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:26:41.230 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:26:41.230 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:560: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:41.420 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:41.420 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 42:135 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:41.420 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:41.420 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:41.420 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:41.429 INFO:tasks.workunit.client.0.trial122.stdout:wait for peering 2026-02-22T23:26:41.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:561: TEST_divergent_2: echo 'wait for peering' 2026-02-22T23:26:41.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:562: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:41.632 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:41.632 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 42:135 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:41.632 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:41.632 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:41.632 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:41.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:563: TEST_divergent_2: rados -p test put foo /etc/fstab 2026-02-22T23:26:41.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:567: TEST_divergent_2: echo 'killing divergent 1' 2026-02-22T23:26:41.670 INFO:tasks.workunit.client.0.trial122.stdout:killing divergent 1 2026-02-22T23:26:41.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:568: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:41.863 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:41.863 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 42:135 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:41.863 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:41.863 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:41.864 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:41.871 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:569: TEST_divergent_2: kill_daemons td/divergent-priors KILL osd.1 2026-02-22T23:26:41.872 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:26:41.872 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:26:41.872 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:26:41.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:26:41.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:26:41.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:572: TEST_divergent_2: expfile=td/divergent-priors/exp.20359.out 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:573: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20359.out 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-22T23:26:41.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20359.out 2026-02-22T23:26:42.632 INFO:tasks.workunit.client.0.trial122.stderr:Exporting 1.0 info 1.0( v 47'102 lc 0'0 (26'90,47'102] local-lis/les=45/46 n=101 ec=20/20 lis/c=45/20 les/c/f=46/21/0 sis=45) 2026-02-22T23:26:42.633 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-22T23:26:42.633 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-22T23:26:42.633 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-22T23:26:42.633 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:091f9f1d:::existing_94:head# 2026-02-22T23:26:42.633 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:0a865a93:::existing_54:head# 2026-02-22T23:26:42.633 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:0fed513c:::existing_41:head# 2026-02-22T23:26:42.634 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:13b9e0b8:::existing_92:head# 2026-02-22T23:26:42.634 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:169157fa:::existing_81:head# 2026-02-22T23:26:42.634 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:17731382:::existing_58:head# 2026-02-22T23:26:42.634 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:18a45a39:::existing_43:head# 2026-02-22T23:26:42.634 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:1c160f01:::existing_66:head# 2026-02-22T23:26:42.634 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:1dd6c249:::existing_97:head# 2026-02-22T23:26:42.635 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:227e0c77:::existing_7:head# 2026-02-22T23:26:42.635 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:23e1b063:::existing_59:head# 2026-02-22T23:26:42.635 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:28566edb:::existing_78:head# 2026-02-22T23:26:42.635 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:2a101182:::existing_20:head# 2026-02-22T23:26:42.635 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:2d1519da:::existing_96:head# 2026-02-22T23:26:42.636 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:3078b87a:::existing_86:head# 2026-02-22T23:26:42.636 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:30bde01b:::existing_15:head# 2026-02-22T23:26:42.636 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:317a7e19:::existing_56:head# 2026-02-22T23:26:42.636 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:38b9c7fa:::existing_33:head# 2026-02-22T23:26:42.636 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:3e16d84b:::existing_48:head# 2026-02-22T23:26:42.637 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:45fe8203:::existing_75:head# 2026-02-22T23:26:42.637 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:4e82c9ba:::existing_50:head# 2026-02-22T23:26:42.637 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:5298dcdb:::existing_57:head# 2026-02-22T23:26:42.637 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:52ebaa44:::existing_32:head# 2026-02-22T23:26:42.637 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:53588912:::existing_9:head# 2026-02-22T23:26:42.637 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:5b7e2de6:::existing_84:head# 2026-02-22T23:26:42.638 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:5da0255c:::existing_13:head# 2026-02-22T23:26:42.638 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:5da2dab8:::existing_53:head# 2026-02-22T23:26:42.638 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:5ee3514b:::existing_30:head# 2026-02-22T23:26:42.638 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:602f83fe:::foo:head# 2026-02-22T23:26:42.638 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:60426cc7:::existing_62:head# 2026-02-22T23:26:42.639 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:628918ca:::existing_79:head# 2026-02-22T23:26:42.639 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:63f8ab2d:::existing_77:head# 2026-02-22T23:26:42.639 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:67e1780f:::existing_27:head# 2026-02-22T23:26:42.639 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:68143ca1:::existing_71:head# 2026-02-22T23:26:42.639 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:6bac9509:::existing_29:head# 2026-02-22T23:26:42.639 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:6bbec798:::existing_45:head# 2026-02-22T23:26:42.640 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:6cb23a40:::existing_65:head# 2026-02-22T23:26:42.640 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:7055ec5e:::existing_83:head# 2026-02-22T23:26:42.640 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:70a54e2b:::existing_87:head# 2026-02-22T23:26:42.640 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:750ace21:::existing_19:head# 2026-02-22T23:26:42.640 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:78623b1d:::existing_6:head# 2026-02-22T23:26:42.641 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:79881152:::existing_63:head# 2026-02-22T23:26:42.641 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:7a80c0d5:::existing_99:head# 2026-02-22T23:26:42.641 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:81490677:::existing_76:head# 2026-02-22T23:26:42.641 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:85ec8e5d:::existing_85:head# 2026-02-22T23:26:42.641 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:8e2abbf0:::existing_49:head# 2026-02-22T23:26:42.642 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:8ee0fe0f:::existing_52:head# 2026-02-22T23:26:42.642 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:923c5d4d:::existing_5:head# 2026-02-22T23:26:42.642 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:92731e71:::existing_89:head# 2026-02-22T23:26:42.642 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:928205cf:::existing_82:head# 2026-02-22T23:26:42.642 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:93be2267:::existing_24:head# 2026-02-22T23:26:42.642 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:99ceda2c:::existing_44:head# 2026-02-22T23:26:42.643 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:9dd917ee:::existing_26:head# 2026-02-22T23:26:42.643 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a28805a6:::existing_72:head# 2026-02-22T23:26:42.643 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a294a8f6:::existing_16:head# 2026-02-22T23:26:42.643 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a2f5473a:::existing_42:head# 2026-02-22T23:26:42.643 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a504a533:::existing_90:head# 2026-02-22T23:26:42.644 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a691337e:::existing_61:head# 2026-02-22T23:26:42.644 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a6e8d7ac:::existing_28:head# 2026-02-22T23:26:42.644 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:a9fde86e:::existing_18:head# 2026-02-22T23:26:42.644 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:aa309faa:::existing_68:head# 2026-02-22T23:26:42.644 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:ae40b0ed:::existing_31:head# 2026-02-22T23:26:42.644 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:b279b1fe:::existing_10:head# 2026-02-22T23:26:42.645 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:b343e277:::existing_88:head# 2026-02-22T23:26:42.645 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:b7130e90:::existing_80:head# 2026-02-22T23:26:42.645 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:bc6f7309:::existing_70:head# 2026-02-22T23:26:42.645 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:be8489b7:::existing_38:head# 2026-02-22T23:26:42.645 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:beb4e5e3:::existing_47:head# 2026-02-22T23:26:42.646 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:bebec877:::existing_74:head# 2026-02-22T23:26:42.646 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:c04347d8:::existing_60:head# 2026-02-22T23:26:42.646 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:c156033e:::existing_100:head# 2026-02-22T23:26:42.646 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:c19f5a8e:::existing_8:head# 2026-02-22T23:26:42.646 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:c3e6ebe4:::existing_17:head# 2026-02-22T23:26:42.646 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:c6c5a7c2:::existing_39:head# 2026-02-22T23:26:42.647 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:cc1d0abf:::existing_95:head# 2026-02-22T23:26:42.647 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:ccb3865b:::existing_73:head# 2026-02-22T23:26:42.647 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:cd84e19e:::existing_40:head# 2026-02-22T23:26:42.647 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:d2cd64ce:::existing_37:head# 2026-02-22T23:26:42.647 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:d41a25c9:::existing_22:head# 2026-02-22T23:26:42.648 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:d7f088da:::existing_91:head# 2026-02-22T23:26:42.648 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:d9cbd1d6:::existing_64:head# 2026-02-22T23:26:42.648 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:dcc4ff82:::existing_98:head# 2026-02-22T23:26:42.648 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:dd677bd3:::existing_35:head# 2026-02-22T23:26:42.648 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:dd6d90e7:::existing_46:head# 2026-02-22T23:26:42.648 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:e0e22077:::existing_21:head# 2026-02-22T23:26:42.649 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:e255b710:::existing_55:head# 2026-02-22T23:26:42.649 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:eff1d5db:::existing_14:head# 2026-02-22T23:26:42.649 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:f1b156bf:::existing_23:head# 2026-02-22T23:26:42.649 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:f5ce4199:::existing_34:head# 2026-02-22T23:26:42.649 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:f6809456:::existing_11:head# 2026-02-22T23:26:42.650 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:f699a2a9:::existing_67:head# 2026-02-22T23:26:42.650 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:f9baac43:::existing_25:head# 2026-02-22T23:26:42.650 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:fb76688e:::existing_51:head# 2026-02-22T23:26:42.650 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:fbb92eec:::existing_69:head# 2026-02-22T23:26:42.650 INFO:tasks.workunit.client.0.trial122.stderr:Export successful 2026-02-22T23:26:42.650 INFO:tasks.workunit.client.0.trial122.stdout: marking collection for removal 2026-02-22T23:26:42.651 INFO:tasks.workunit.client.0.trial122.stdout:setting '_remove' omap key 2026-02-22T23:26:42.651 INFO:tasks.workunit.client.0.trial122.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-22T23:26:42.652 INFO:tasks.workunit.client.0.trial122.stderr:Remove successful 2026-02-22T23:26:43.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:574: TEST_divergent_2: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.20359.out 2026-02-22T23:26:43.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:26:43.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:26:43.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-22T23:26:43.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:26:43.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-22T23:26:43.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.20359.out 2026-02-22T23:26:43.830 INFO:tasks.workunit.client.0.trial122.stdout:get_pg_num_history pg_num_history pg_num_history(e47 pg_nums {1={20=1}} deleted_pools ) 2026-02-22T23:26:43.831 INFO:tasks.workunit.client.0.trial122.stdout:Importing pgid 1.0 2026-02-22T23:26:43.831 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-22T23:26:43.831 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.831 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-22T23:26:43.831 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.833 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-22T23:26:43.833 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.833 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:091f9f1d:::existing_94:head# 2026-02-22T23:26:43.833 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:0a865a93:::existing_54:head# 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:0fed513c:::existing_41:head# 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:13b9e0b8:::existing_92:head# 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:169157fa:::existing_81:head# 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:17731382:::existing_58:head# 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:18a45a39:::existing_43:head# 2026-02-22T23:26:43.834 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.835 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:1c160f01:::existing_66:head# 2026-02-22T23:26:43.835 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:1dd6c249:::existing_97:head# 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:227e0c77:::existing_7:head# 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:23e1b063:::existing_59:head# 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:28566edb:::existing_78:head# 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:2a101182:::existing_20:head# 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.838 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:2d1519da:::existing_96:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:3078b87a:::existing_86:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:30bde01b:::existing_15:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:317a7e19:::existing_56:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:38b9c7fa:::existing_33:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:3e16d84b:::existing_48:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:45fe8203:::existing_75:head# 2026-02-22T23:26:43.839 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:4e82c9ba:::existing_50:head# 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:5298dcdb:::existing_57:head# 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:52ebaa44:::existing_32:head# 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:53588912:::existing_9:head# 2026-02-22T23:26:43.840 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:5b7e2de6:::existing_84:head# 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:5da0255c:::existing_13:head# 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:5da2dab8:::existing_53:head# 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:5ee3514b:::existing_30:head# 2026-02-22T23:26:43.844 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:602f83fe:::foo:head# 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:60426cc7:::existing_62:head# 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:628918ca:::existing_79:head# 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:63f8ab2d:::existing_77:head# 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:67e1780f:::existing_27:head# 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:68143ca1:::existing_71:head# 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.845 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:6bac9509:::existing_29:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:6bbec798:::existing_45:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:6cb23a40:::existing_65:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:7055ec5e:::existing_83:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:70a54e2b:::existing_87:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:750ace21:::existing_19:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:78623b1d:::existing_6:head# 2026-02-22T23:26:43.846 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:79881152:::existing_63:head# 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:7a80c0d5:::existing_99:head# 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:81490677:::existing_76:head# 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:85ec8e5d:::existing_85:head# 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:8e2abbf0:::existing_49:head# 2026-02-22T23:26:43.847 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.852 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:8ee0fe0f:::existing_52:head# 2026-02-22T23:26:43.852 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.852 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:923c5d4d:::existing_5:head# 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:92731e71:::existing_89:head# 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:928205cf:::existing_82:head# 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:93be2267:::existing_24:head# 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:99ceda2c:::existing_44:head# 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:9dd917ee:::existing_26:head# 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.853 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a28805a6:::existing_72:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a294a8f6:::existing_16:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a2f5473a:::existing_42:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a504a533:::existing_90:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a691337e:::existing_61:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a6e8d7ac:::existing_28:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:a9fde86e:::existing_18:head# 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.854 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:aa309faa:::existing_68:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:ae40b0ed:::existing_31:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:b279b1fe:::existing_10:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:b343e277:::existing_88:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:b7130e90:::existing_80:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:bc6f7309:::existing_70:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:be8489b7:::existing_38:head# 2026-02-22T23:26:43.855 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:beb4e5e3:::existing_47:head# 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:bebec877:::existing_74:head# 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:c04347d8:::existing_60:head# 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:c156033e:::existing_100:head# 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:c19f5a8e:::existing_8:head# 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:c3e6ebe4:::existing_17:head# 2026-02-22T23:26:43.856 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.857 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:c6c5a7c2:::existing_39:head# 2026-02-22T23:26:43.857 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.857 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:cc1d0abf:::existing_95:head# 2026-02-22T23:26:43.857 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:ccb3865b:::existing_73:head# 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:cd84e19e:::existing_40:head# 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:d2cd64ce:::existing_37:head# 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:d41a25c9:::existing_22:head# 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:d7f088da:::existing_91:head# 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.861 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:d9cbd1d6:::existing_64:head# 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:dcc4ff82:::existing_98:head# 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:dd677bd3:::existing_35:head# 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:dd6d90e7:::existing_46:head# 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:e0e22077:::existing_21:head# 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:e255b710:::existing_55:head# 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.862 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:eff1d5db:::existing_14:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:f1b156bf:::existing_23:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:f5ce4199:::existing_34:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:f6809456:::existing_11:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:f699a2a9:::existing_67:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:f9baac43:::existing_25:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:fb76688e:::existing_51:head# 2026-02-22T23:26:43.863 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.864 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:fbb92eec:::existing_69:head# 2026-02-22T23:26:43.864 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:26:43.864 INFO:tasks.workunit.client.0.trial122.stdout:write_pg epoch 47 info 1.0( v 47'102 lc 0'0 (26'90,47'102] local-lis/les=45/46 n=101 ec=20/20 lis/c=45/20 les/c/f=46/21/0 sis=45) 2026-02-22T23:26:43.864 INFO:tasks.workunit.client.0.trial122.stdout:Import successful 2026-02-22T23:26:44.393 INFO:tasks.workunit.client.0.trial122.stdout:reviving divergent 1 2026-02-22T23:26:44.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:576: TEST_divergent_2: echo 'reviving divergent 1' 2026-02-22T23:26:44.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:577: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:26:44.580 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:26:44.580 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 100 0 0 73039 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:25:33.662052+0000 40'101 42:135 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:52:50.732693+0000 0 0 2026-02-22T23:26:44.580 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:26:44.580 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:26:44.581 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:26:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:578: TEST_divergent_2: activate_osd td/divergent-priors 1 2026-02-22T23:26:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:26:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:26:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:26:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:26:44.592 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:26:44.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:26:44.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:26:44.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:26:44.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:26:44.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:26:44.594 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:26:44.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:26:44.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:26:44.595 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:26:44.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:26:44.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:26:44.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:26:44.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:26:44.618 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:44.612+0000 7fbff2f8e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:26:44.619 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:44.612+0000 7fbff2f8e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:26:44.620 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:44.616+0000 7fbff2f8e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:26:44.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:26:44.797 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:26:44.797 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:26:44.797 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:26:44.997 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/360547886,v1:127.0.0.1:6819/360547886] [v2:127.0.0.1:6820/360547886,v1:127.0.0.1:6821/360547886] exists,up 1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:579: TEST_divergent_2: wait_for_osd up 1 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:26:44.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:26:44.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:26:44.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:26:44.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:26:44.999 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:26:44.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:26:45.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:26:45.191 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 45 up_thru 45 down_at 28 last_clean_interval [9,26) [v2:127.0.0.1:6818/360547886,v1:127.0.0.1:6819/360547886] [v2:127.0.0.1:6820/360547886,v1:127.0.0.1:6821/360547886] exists,up 1a5e6140-34fa-45e2-902a-503c534acc20 2026-02-22T23:26:45.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:26:45.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:26:45.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:26:45.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:581: TEST_divergent_2: sleep 20 2026-02-22T23:26:46.145 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:46.140+0000 7fbff2f8e8c0 -1 Falling back to public interface 2026-02-22T23:26:49.133 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:26:49.128+0000 7fbff2f8e8c0 -1 osd.1 47 log_to_monitors true 2026-02-22T23:27:05.193 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: get_asok_path osd.1 2026-02-22T23:27:05.194 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:27:05.194 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:27:05.195 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:27:05.195 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:05.195 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:05.195 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:27:05.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: CEPH_ARGS= 2026-02-22T23:27:05.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:582: TEST_divergent_2: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok dump_ops_in_flight 2026-02-22T23:27:05.257 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:27:05.257 INFO:tasks.workunit.client.0.trial122.stdout: "ops": [], 2026-02-22T23:27:05.257 INFO:tasks.workunit.client.0.trial122.stdout: "num_ops": 0 2026-02-22T23:27:05.257 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:27:05.262 INFO:tasks.workunit.client.0.trial122.stdout:allowing recovery 2026-02-22T23:27:05.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:584: TEST_divergent_2: echo 'allowing recovery' 2026-02-22T23:27:05.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:585: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:27:05.459 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:27:05.460 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73767 0 0 12 90 12 active+clean 2026-02-22T23:26:50.388916+0000 47'102 51:164 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:33:54.318001+0000 0 0 2026-02-22T23:27:05.460 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:27:05.460 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:27:05.460 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:27:05.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-22T23:27:05.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-22T23:27:05.550 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:27:05.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-22T23:27:05.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-22T23:27:05.634 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:27:05.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:587: TEST_divergent_2: for i in $osds 2026-02-22T23:27:05.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:589: TEST_divergent_2: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-22T23:27:05.721 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:27:05.727 INFO:tasks.workunit.client.0.trial122.stdout:reading divergent objects 2026-02-22T23:27:05.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:592: TEST_divergent_2: echo 'reading divergent objects' 2026-02-22T23:27:05.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:593: TEST_divergent_2: ceph pg dump pgs 2026-02-22T23:27:05.920 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:27:05.920 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73767 0 0 12 90 12 active+clean 2026-02-22T23:26:50.388916+0000 47'102 51:164 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:24:57.053324+0000 0'0 2026-02-22T23:24:57.053324+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:33:54.318001+0000 0 0 2026-02-22T23:27:05.920 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:27:05.921 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:27:05.921 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:27:05.929 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: expr 2 + 2 2026-02-22T23:27:05.931 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: seq 1 4 2026-02-22T23:27:05.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:27:05.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_1 td/divergent-priors/existing 2026-02-22T23:27:05.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:27:05.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_2 td/divergent-priors/existing 2026-02-22T23:27:05.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:27:05.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_3 td/divergent-priors/existing 2026-02-22T23:27:06.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:594: TEST_divergent_2: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:27:06.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:596: TEST_divergent_2: rados -p test get existing_4 td/divergent-priors/existing 2026-02-22T23:27:06.032 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: seq 1 2 2026-02-22T23:27:06.033 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:27:06.033 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-22T23:27:06.052 INFO:tasks.workunit.client.0.trial122.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-22T23:27:06.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:598: TEST_divergent_2: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:27:06.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:600: TEST_divergent_2: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-22T23:27:06.080 INFO:tasks.workunit.client.0.trial122.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-22T23:27:06.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:602: TEST_divergent_2: rm -f td/divergent-priors/existing 2026-02-22T23:27:06.084 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-22T23:27:06.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:604: TEST_divergent_2: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:27:06.088 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 26'106 (26'4) delete 1:70b3bf12:::existing_4:head by client.4565.0:1 2026-02-22T23:25:10.775606+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:27:06.088 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: keeping 26'106 (26'4) delete 1:70b3bf12:::existing_4:head by client.4565.0:1 2026-02-22T23:25:10.775606+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:27:06.088 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:27:06.088 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 26'4 first_divergent_update: 26'106 last_divergent_update: 26'106 2026-02-22T23:27:06.088 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: more recent entry found: 40'101 (26'4) modify 1:70b3bf12:::existing_4:head by client.4688.0:1 2026-02-22T23:26:03.937376+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false, already merged 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 26'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4552.0:1 2026-02-22T23:25:10.768212+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: keeping 26'101 (0'0) modify 1:75ac5091:::newobject_2:head by client.4552.0:1 2026-02-22T23:25:10.768212+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 26'101 last_divergent_update: 26'101 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 26'105 (26'3) delete 1:8881e6d8:::existing_3:head by client.4562.0:1 2026-02-22T23:25:10.773698+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: keeping 26'105 (26'3) delete 1:8881e6d8:::existing_3:head by client.4562.0:1 2026-02-22T23:25:10.773698+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 26'3 first_divergent_update: 26'105 last_divergent_update: 26'105 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-22T23:27:06.089 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 26'106 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 26'105 (26'3) delete 1:8881e6d8:::existing_3:head by client.4562.0:1 2026-02-22T23:25:10.773698+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 26'3 <= info.log_tail 26'90 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 26'104 (26'2) modify 1:97db2a0f:::existing_2:head by client.4559.0:1 2026-02-22T23:25:10.771466+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: keeping 26'104 (26'2) modify 1:97db2a0f:::existing_2:head by client.4559.0:1 2026-02-22T23:25:10.771466+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 26'2 first_divergent_update: 26'104 last_divergent_update: 26'104 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 26'106 2026-02-22T23:27:06.090 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 26'104 (26'2) modify 1:97db2a0f:::existing_2:head by client.4559.0:1 2026-02-22T23:25:10.771466+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:27:06.091 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-22T23:27:06.091 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 26'2 <= info.log_tail 26'90 2026-02-22T23:27:06.091 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 26'103 (26'1) modify 1:d858c035:::existing_1:head by client.4558.0:1 2026-02-22T23:25:10.771228+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:27:06.091 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: keeping 26'103 (26'1) modify 1:d858c035:::existing_1:head by client.4558.0:1 2026-02-22T23:25:10.771228+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:27:06.091 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:27:06.091 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 26'1 first_divergent_update: 26'103 last_divergent_update: 26'103 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 26'106 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 26'103 (26'1) modify 1:d858c035:::existing_1:head by client.4558.0:1 2026-02-22T23:25:10.771228+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 26'1 <= info.log_tail 26'90 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 26'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4553.0:1 2026-02-22T23:25:10.768470+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 20 _merge_object_divergent_entries: keeping 26'102 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4553.0:1 2026-02-22T23:25:10.768470+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 26'102 last_divergent_update: 26'102 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-22T23:27:06.092 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:26:40.608+0000 7f6ad7a85640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-22T23:27:06.093 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-22T23:27:06.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:606: TEST_divergent_2: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:27:06.093 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: find td/divergent-priors -name '*osd*log' 2026-02-22T23:27:06.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:612: TEST_divergent_2: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:27:06.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:617: TEST_divergent_2: echo success 2026-02-22T23:27:06.097 INFO:tasks.workunit.client.0.trial122.stdout:success 2026-02-22T23:27:06.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:619: TEST_divergent_2: rm td/divergent-priors/td/divergent-priors/exp.20359.out 2026-02-22T23:27:06.098 INFO:tasks.workunit.client.0.trial122.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.20359.out': No such file or directory 2026-02-22T23:27:06.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:621: TEST_divergent_2: delete_pool test 2026-02-22T23:27:06.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-22T23:27:06.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-22T23:27:06.342 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' does not exist 2026-02-22T23:27:06.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:622: TEST_divergent_2: kill_daemons td/divergent-priors 2026-02-22T23:27:06.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:27:06.353 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:27:06.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:27:06.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:27:06.354 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:27:14.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:27:14.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-22T23:27:14.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:27:14.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:27:14.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:27:14.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:27:14.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:27:14.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:27:14.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:27:14.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:27:14.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:27:14.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:27:14.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:27:14.681 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:27:14.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:27:14.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:27:14.682 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:27:14.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:27:14.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:27:14.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:27:14.684 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:27:14.685 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:27:14.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:27:14.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:27:14.707 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:27:14.707 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:14.707 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:14.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:27:14.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:27:14.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:27:14.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:27:14.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:27:14.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:27:14.712 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:27:14.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:27:14.713 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:27:14.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:27:14.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:27:14.715 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:27:14.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:27:14.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:27:14.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:27:14.716 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:27:14.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:27:14.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:27:14.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:27:14.720 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:27:14.721 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:14.721 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:14.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:27:14.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:27:14.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:27:14.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-22T23:27:14.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:27:14.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:14.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:14.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20359 2026-02-22T23:27:14.724 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_3 td/divergent-priors 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:628: TEST_divergent_3: local dir=td/divergent-priors 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:631: TEST_divergent_3: local dummyfile=/etc/fstab 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:632: TEST_divergent_3: local dummyfile2=/etc/resolv.conf 2026-02-22T23:27:14.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:634: TEST_divergent_3: local num_osds=3 2026-02-22T23:27:14.727 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: expr 3 - 1 2026-02-22T23:27:14.727 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: seq 0 2 2026-02-22T23:27:14.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:635: TEST_divergent_3: local 'osds=0 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:636: TEST_divergent_3: run_mon td/divergent-priors a 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-22T23:27:14.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-22T23:27:14.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:27:14.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:27:14.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:27:14.752 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:27:14.752 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:14.752 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:14.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:14.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:27:14.782 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:27:14.783 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:27:14.783 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:27:14.783 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:27:14.783 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:27:14.784 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:27:14.785 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:27:14.786 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get fsid 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:27:14.840 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:27:14.841 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:27:14.841 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:27:14.841 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:14.842 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:14.842 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:27:14.842 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:27:14.842 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get mon_host 2026-02-22T23:27:14.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:637: TEST_divergent_3: run_mgr td/divergent-priors x 2026-02-22T23:27:14.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-22T23:27:14.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:27:14.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:27:14.896 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:27:14.896 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-22T23:27:14.896 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:15.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:15.005 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:27:15.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 0 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:27:15.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:27:15.027 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:27:15.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:27:15.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:27:15.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:27:15.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:27:15.029 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:27:15.030 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=a9320b9d-2589-436d-a8bb-32a33f236ef9 2026-02-22T23:27:15.030 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 a9320b9d-2589-436d-a8bb-32a33f236ef9 2026-02-22T23:27:15.030 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 a9320b9d-2589-436d-a8bb-32a33f236ef9' 2026-02-22T23:27:15.030 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:27:15.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDTkJtpMtqLAhAAKGZ1gH/Zih8WcWdFnk/QpA== 2026-02-22T23:27:15.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDTkJtpMtqLAhAAKGZ1gH/Zih8WcWdFnk/QpA=="}' 2026-02-22T23:27:15.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new a9320b9d-2589-436d-a8bb-32a33f236ef9 -i td/divergent-priors/0/new.json 2026-02-22T23:27:15.156 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:27:15.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-22T23:27:15.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDTkJtpMtqLAhAAKGZ1gH/Zih8WcWdFnk/QpA== --osd-uuid a9320b9d-2589-436d-a8bb-32a33f236ef9 2026-02-22T23:27:15.184 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:15.179+0000 7fd2358948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:15.185 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:15.179+0000 7fd2358948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:15.186 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:15.179+0000 7fd2358948c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:15.187 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:15.179+0000 7fd2358948c0 -1 bdev(0x55c8bdbcdc00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:27:15.187 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:15.179+0000 7fd2358948c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-22T23:27:17.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-22T23:27:17.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:27:17.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:27:17.798 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:27:17.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:27:18.072 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:27:18.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:27:18.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:27:18.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:27:18.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:27:18.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:27:18.099 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:18.091+0000 7f33ee2848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:18.100 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:18.095+0000 7f33ee2848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:18.101 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:18.095+0000 7f33ee2848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:18.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:27:18.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:27:18.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:27:18.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:27:18.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:27:18.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:18.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:27:18.276 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:27:18.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:18.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:27:18.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:19.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:19.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:19.477 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:27:19.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:27:19.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:19.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:27:19.593 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:19.591+0000 7f33ee2848c0 -1 Falling back to public interface 2026-02-22T23:27:19.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:20.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:20.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:20.672 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:27:20.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:27:20.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:20.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:27:20.866 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:21.867 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:21.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:21.868 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:27:21.869 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:27:21.869 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:21.869 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:27:22.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:22.835 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:22.831+0000 7f33ee2848c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:27:23.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:23.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:23.065 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:27:23.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:27:23.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:23.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:27:23.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:24.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:24.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:24.272 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:27:24.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:27:24.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:24.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:27:24.476 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2829096372,v1:127.0.0.1:6803/2829096372] [v2:127.0.0.1:6804/2829096372,v1:127.0.0.1:6805/2829096372] exists,up a9320b9d-2589-436d-a8bb-32a33f236ef9 2026-02-22T23:27:24.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:27:24.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:27:24.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:27:24.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-22T23:27:24.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 1 2026-02-22T23:27:24.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:27:24.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:27:24.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:27:24.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:27:24.484 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:27:24.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:27:24.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 cc902fd6-3159-44fe-844d-063a2679a371' 2026-02-22T23:27:24.484 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:27:24.485 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:27:24.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDckJtpqbh6HRAAb+YFL0zEM3b9KtfQ5s8XxA== 2026-02-22T23:27:24.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDckJtpqbh6HRAAb+YFL0zEM3b9KtfQ5s8XxA=="}' 2026-02-22T23:27:24.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new cc902fd6-3159-44fe-844d-063a2679a371 -i td/divergent-priors/1/new.json 2026-02-22T23:27:24.706 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:27:24.715 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-22T23:27:24.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDckJtpqbh6HRAAb+YFL0zEM3b9KtfQ5s8XxA== --osd-uuid cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:27:24.734 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:24.731+0000 7fc88859d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:24.736 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:24.735+0000 7fc88859d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:24.737 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:24.735+0000 7fc88859d8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:24.738 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:24.735+0000 7fc88859d8c0 -1 bdev(0x55a20b349c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:27:24.738 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:24.735+0000 7fc88859d8c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-22T23:27:27.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-22T23:27:27.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:27:27.006 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:27:27.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:27:27.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:27:27.268 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:27:27.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:27:27.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:27:27.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:27:27.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:27:27.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:27:27.295 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:27.291+0000 7f7c00b888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:27.297 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:27.295+0000 7f7c00b888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:27.298 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:27.295+0000 7f7c00b888c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:27.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:27:27.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:27:27.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:27:27.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:27:27.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:27:27.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:27.467 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:27:27.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:27:27.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:27.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:27:27.666 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:28.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:28.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:28.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:27:28.668 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:27:28.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:28.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:27:28.793 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:28.791+0000 7f7c00b888c0 -1 Falling back to public interface 2026-02-22T23:27:28.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:29.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:29.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:29.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:27:29.865 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:27:29.866 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:29.866 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:27:30.061 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:31.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:31.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:31.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:27:31.063 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:27:31.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:31.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:27:31.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:31.842 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:31.839+0000 7f7c00b888c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:27:32.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:32.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:32.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:27:32.263 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:27:32.264 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:32.264 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:27:32.464 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1381058531,v1:127.0.0.1:6811/1381058531] [v2:127.0.0.1:6812/1381058531,v1:127.0.0.1:6813/1381058531] exists,up cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:638: TEST_divergent_3: for i in $osds 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:640: TEST_divergent_3: run_osd td/divergent-priors 2 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:27:32.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:27:32.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:27:32.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:27:32.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:27:32.472 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:27:32.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ddddd36a-efe9-46bc-8e09-84f6fdabe0bc 2026-02-22T23:27:32.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 ddddd36a-efe9-46bc-8e09-84f6fdabe0bc' 2026-02-22T23:27:32.472 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 ddddd36a-efe9-46bc-8e09-84f6fdabe0bc 2026-02-22T23:27:32.473 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:27:32.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDkkJtp2/7GHBAAxjFTOFjqTR7k+YXTeZ1QjQ== 2026-02-22T23:27:32.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDkkJtp2/7GHBAAxjFTOFjqTR7k+YXTeZ1QjQ=="}' 2026-02-22T23:27:32.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ddddd36a-efe9-46bc-8e09-84f6fdabe0bc -i td/divergent-priors/2/new.json 2026-02-22T23:27:32.681 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:27:32.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-22T23:27:32.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDkkJtp2/7GHBAAxjFTOFjqTR7k+YXTeZ1QjQ== --osd-uuid ddddd36a-efe9-46bc-8e09-84f6fdabe0bc 2026-02-22T23:27:32.708 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:32.707+0000 7fa32e8ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:32.709 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:32.707+0000 7fa32e8ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:32.711 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:32.707+0000 7fa32e8ef8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:32.711 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:32.711+0000 7fa32e8ef8c0 -1 bdev(0x55bc001cbc00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:27:32.711 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:32.711+0000 7fa32e8ef8c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-22T23:27:34.996 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-22T23:27:34.996 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:27:34.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:27:34.998 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:27:34.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:27:35.270 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:27:35.270 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:27:35.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:27:35.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:27:35.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:27:35.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:27:35.287 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:35.287+0000 7f2f5baec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:35.289 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:35.287+0000 7f2f5baec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:35.289 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:35.287+0000 7f2f5baec8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:27:35.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:27:35.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:27:35.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:27:35.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:27:35.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:27:35.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:35.477 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:27:35.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:27:35.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:35.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:27:35.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:36.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:36.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:36.676 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:27:36.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:27:36.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:36.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:27:36.797 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:36.795+0000 7f2f5baec8c0 -1 Falling back to public interface 2026-02-22T23:27:36.872 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:37.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:37.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:37.874 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:27:37.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:27:37.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:37.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:27:38.069 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:39.070 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:27:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:27:39.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:39.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:27:39.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:40.033 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:27:40.031+0000 7f2f5baec8c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:27:40.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:40.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:40.267 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:27:40.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:27:40.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:40.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:27:40.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:27:41.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:27:41.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:27:41.461 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:27:41.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:27:41.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:27:41.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:27:41.663 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1788134219,v1:127.0.0.1:6819/1788134219] [v2:127.0.0.1:6820/1788134219,v1:127.0.0.1:6821/1788134219] exists,up ddddd36a-efe9-46bc-8e09-84f6fdabe0bc 2026-02-22T23:27:41.664 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:27:41.664 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:27:41.664 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:27:41.664 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:643: TEST_divergent_3: ceph osd set noout 2026-02-22T23:27:41.963 INFO:tasks.workunit.client.0.trial122.stderr:noout is set 2026-02-22T23:27:41.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:644: TEST_divergent_3: ceph osd set noin 2026-02-22T23:27:42.268 INFO:tasks.workunit.client.0.trial122.stderr:noin is set 2026-02-22T23:27:42.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:645: TEST_divergent_3: ceph osd set nodown 2026-02-22T23:27:42.573 INFO:tasks.workunit.client.0.trial122.stderr:nodown is set 2026-02-22T23:27:42.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:646: TEST_divergent_3: create_pool test 1 1 2026-02-22T23:27:42.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 2026-02-22T23:27:42.823 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' already exists 2026-02-22T23:27:42.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:27:43.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:647: TEST_divergent_3: ceph osd pool set test size 3 2026-02-22T23:27:44.088 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 size to 3 2026-02-22T23:27:44.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:648: TEST_divergent_3: ceph osd pool set test min_size 2 2026-02-22T23:27:44.395 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 min_size to 2 2026-02-22T23:27:44.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:651: TEST_divergent_3: ceph osd pool set test pg_autoscale_mode on 2026-02-22T23:27:44.699 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 pg_autoscale_mode to on 2026-02-22T23:27:44.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:653: TEST_divergent_3: divergent=-1 2026-02-22T23:27:44.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: date +%s 2026-02-22T23:27:44.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:654: TEST_divergent_3: start_time=1771802864 2026-02-22T23:27:44.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:655: TEST_divergent_3: max_duration=300 2026-02-22T23:27:44.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: '[' -1 -le -1 ']' 2026-02-22T23:27:44.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:659: TEST_divergent_3: flush_pg_stats 2026-02-22T23:27:44.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:27:44.710 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:27:44.913 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:27:44.913 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:27:44.913 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:27:44.913 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:27:44.913 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:44.913 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:27:44.991 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-22T23:27:44.991 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-22T23:27:44.991 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-22T23:27:44.992 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:44.992 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:27:45.072 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-22T23:27:45.072 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-22T23:27:45.072 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-22T23:27:45.072 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:45.073 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:27:45.148 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-22T23:27:45.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-22T23:27:45.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-60129542147' 2026-02-22T23:27:45.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:45.150 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-22T23:27:45.150 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:45.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:27:45.151 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-22T23:27:45.151 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:45.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-22T23:27:45.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-22T23:27:45.153 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836487 2026-02-22T23:27:45.153 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:27:45.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-22T23:27:45.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:45.360 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-22T23:27:45.360 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:45.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:27:45.362 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-22T23:27:45.362 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:45.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-22T23:27:45.364 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705669 2026-02-22T23:27:45.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-22T23:27:45.364 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:27:45.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-22T23:27:45.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:45.569 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-22T23:27:45.569 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:45.570 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:27:45.570 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-22T23:27:45.571 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:45.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-22T23:27:45.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-22T23:27:45.572 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542147 2026-02-22T23:27:45.573 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:27:45.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-22T23:27:45.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:660: TEST_divergent_3: wait_for_clean 2026-02-22T23:27:45.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:27:45.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:27:45.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:27:45.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:27:45.775 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:27:45.775 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:27:45.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:27:45.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:27:45.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:27:45.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:27:45.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:27:45.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:27:45.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:27:45.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:27:45.879 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:27:46.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:27:46.086 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:27:46.086 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:27:46.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:27:46.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:46.086 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:27:46.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-22T23:27:46.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-22T23:27:46.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-22T23:27:46.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:46.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:27:46.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705671 2026-02-22T23:27:46.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705671 2026-02-22T23:27:46.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671' 2026-02-22T23:27:46.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:46.241 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:27:46.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-22T23:27:46.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-22T23:27:46.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705671 2-60129542150' 2026-02-22T23:27:46.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:46.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-22T23:27:46.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:46.321 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:27:46.322 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-22T23:27:46.322 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:46.323 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-22T23:27:46.323 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-22T23:27:46.323 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836489 2026-02-22T23:27:46.324 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:27:46.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836489 2026-02-22T23:27:46.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:27:47.521 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:27:47.521 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:27:47.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-22T23:27:47.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:47.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705671 2026-02-22T23:27:47.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:47.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:27:47.725 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705671 2026-02-22T23:27:47.725 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:47.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705671 2026-02-22T23:27:47.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705671' 2026-02-22T23:27:47.727 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705671 2026-02-22T23:27:47.727 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:27:47.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705671 -lt 38654705671 2026-02-22T23:27:47.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:47.933 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-22T23:27:47.933 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:47.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:27:47.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-22T23:27:47.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:47.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-22T23:27:47.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-22T23:27:47.937 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542150 2026-02-22T23:27:47.937 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:27:48.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-22T23:27:48.135 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:27:48.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:27:48.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:27:48.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:27:48.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:27:48.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:27:48.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:27:48.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:27:48.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:27:48.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:27:48.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:27:48.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:27:48.589 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:27:48.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:27:48.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:27:48.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:27:48.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:27:48.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:27:48.857 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: ceph pg dump pgs --format=json 2026-02-22T23:27:48.857 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: jq '.pg_stats[0].up_primary' 2026-02-22T23:27:49.042 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:27:49.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:663: TEST_divergent_3: divergent=1 2026-02-22T23:27:49.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:664: TEST_divergent_3: echo 'primary and soon to be divergent is 1' 2026-02-22T23:27:49.052 INFO:tasks.workunit.client.0.trial122.stdout:primary and soon to be divergent is 1 2026-02-22T23:27:49.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:665: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:27:49.237 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:27:49.238 INFO:tasks.workunit.client.0.trial122.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-22T23:27:44.137643+0000 0'0 30:23 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:16:26.397145+0000 0 0 2026-02-22T23:27:49.238 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:27:49.238 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:27:49.238 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:27:49.246 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: date +%s 2026-02-22T23:27:49.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:667: TEST_divergent_3: current_time=1771802869 2026-02-22T23:27:49.247 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:668: TEST_divergent_3: expr 1771802869 - 1771802864 2026-02-22T23:27:49.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:668: TEST_divergent_3: elapsed_time=5 2026-02-22T23:27:49.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:669: TEST_divergent_3: '[' 5 -gt 300 ']' 2026-02-22T23:27:49.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:657: TEST_divergent_3: '[' 1 -le -1 ']' 2026-02-22T23:27:49.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:675: TEST_divergent_3: local non_divergent= 2026-02-22T23:27:49.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 0 = 1 ']' 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: non_divergent=' 0' 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 1 = 1 ']' 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:679: TEST_divergent_3: continue 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:676: TEST_divergent_3: for i in $osds 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:678: TEST_divergent_3: '[' 2 = 1 ']' 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:681: TEST_divergent_3: non_divergent=' 0 2' 2026-02-22T23:27:49.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:684: TEST_divergent_3: echo 'writing initial objects' 2026-02-22T23:27:49.251 INFO:tasks.workunit.client.0.trial122.stdout:writing initial objects 2026-02-22T23:27:49.251 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: seq 1 100 2026-02-22T23:27:49.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_1 /etc/fstab 2026-02-22T23:27:49.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_2 /etc/fstab 2026-02-22T23:27:49.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_3 /etc/fstab 2026-02-22T23:27:49.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_4 /etc/fstab 2026-02-22T23:27:49.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_5 /etc/fstab 2026-02-22T23:27:49.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_6 /etc/fstab 2026-02-22T23:27:49.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_7 /etc/fstab 2026-02-22T23:27:49.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_8 /etc/fstab 2026-02-22T23:27:49.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_9 /etc/fstab 2026-02-22T23:27:49.500 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.501 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_10 /etc/fstab 2026-02-22T23:27:49.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_11 /etc/fstab 2026-02-22T23:27:49.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_12 /etc/fstab 2026-02-22T23:27:49.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_13 /etc/fstab 2026-02-22T23:27:49.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_14 /etc/fstab 2026-02-22T23:27:49.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_15 /etc/fstab 2026-02-22T23:27:49.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_16 /etc/fstab 2026-02-22T23:27:49.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_17 /etc/fstab 2026-02-22T23:27:49.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_18 /etc/fstab 2026-02-22T23:27:49.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_19 /etc/fstab 2026-02-22T23:27:49.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_20 /etc/fstab 2026-02-22T23:27:49.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.808 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_21 /etc/fstab 2026-02-22T23:27:49.835 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.835 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_22 /etc/fstab 2026-02-22T23:27:49.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_23 /etc/fstab 2026-02-22T23:27:49.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_24 /etc/fstab 2026-02-22T23:27:49.919 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.919 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_25 /etc/fstab 2026-02-22T23:27:49.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_26 /etc/fstab 2026-02-22T23:27:49.977 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:49.977 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_27 /etc/fstab 2026-02-22T23:27:50.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_28 /etc/fstab 2026-02-22T23:27:50.032 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.033 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_29 /etc/fstab 2026-02-22T23:27:50.061 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.061 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_30 /etc/fstab 2026-02-22T23:27:50.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_31 /etc/fstab 2026-02-22T23:27:50.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_32 /etc/fstab 2026-02-22T23:27:50.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_33 /etc/fstab 2026-02-22T23:27:50.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_34 /etc/fstab 2026-02-22T23:27:50.201 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.201 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_35 /etc/fstab 2026-02-22T23:27:50.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_36 /etc/fstab 2026-02-22T23:27:50.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_37 /etc/fstab 2026-02-22T23:27:50.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_38 /etc/fstab 2026-02-22T23:27:50.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_39 /etc/fstab 2026-02-22T23:27:50.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.333 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_40 /etc/fstab 2026-02-22T23:27:50.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_41 /etc/fstab 2026-02-22T23:27:50.386 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.386 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_42 /etc/fstab 2026-02-22T23:27:50.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_43 /etc/fstab 2026-02-22T23:27:50.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_44 /etc/fstab 2026-02-22T23:27:50.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_45 /etc/fstab 2026-02-22T23:27:50.501 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.501 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_46 /etc/fstab 2026-02-22T23:27:50.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_47 /etc/fstab 2026-02-22T23:27:50.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_48 /etc/fstab 2026-02-22T23:27:50.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_49 /etc/fstab 2026-02-22T23:27:50.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_50 /etc/fstab 2026-02-22T23:27:50.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_51 /etc/fstab 2026-02-22T23:27:50.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_52 /etc/fstab 2026-02-22T23:27:50.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_53 /etc/fstab 2026-02-22T23:27:50.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_54 /etc/fstab 2026-02-22T23:27:50.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_55 /etc/fstab 2026-02-22T23:27:50.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_56 /etc/fstab 2026-02-22T23:27:50.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_57 /etc/fstab 2026-02-22T23:27:50.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_58 /etc/fstab 2026-02-22T23:27:50.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_59 /etc/fstab 2026-02-22T23:27:50.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_60 /etc/fstab 2026-02-22T23:27:50.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_61 /etc/fstab 2026-02-22T23:27:50.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_62 /etc/fstab 2026-02-22T23:27:50.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_63 /etc/fstab 2026-02-22T23:27:50.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:50.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_64 /etc/fstab 2026-02-22T23:27:51.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_65 /etc/fstab 2026-02-22T23:27:51.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_66 /etc/fstab 2026-02-22T23:27:51.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_67 /etc/fstab 2026-02-22T23:27:51.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_68 /etc/fstab 2026-02-22T23:27:51.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_69 /etc/fstab 2026-02-22T23:27:51.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_70 /etc/fstab 2026-02-22T23:27:51.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_71 /etc/fstab 2026-02-22T23:27:51.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_72 /etc/fstab 2026-02-22T23:27:51.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_73 /etc/fstab 2026-02-22T23:27:51.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_74 /etc/fstab 2026-02-22T23:27:51.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_75 /etc/fstab 2026-02-22T23:27:51.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_76 /etc/fstab 2026-02-22T23:27:51.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_77 /etc/fstab 2026-02-22T23:27:51.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_78 /etc/fstab 2026-02-22T23:27:51.372 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.373 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_79 /etc/fstab 2026-02-22T23:27:51.401 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.401 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_80 /etc/fstab 2026-02-22T23:27:51.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_81 /etc/fstab 2026-02-22T23:27:51.456 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_82 /etc/fstab 2026-02-22T23:27:51.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.485 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_83 /etc/fstab 2026-02-22T23:27:51.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_84 /etc/fstab 2026-02-22T23:27:51.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_85 /etc/fstab 2026-02-22T23:27:51.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.569 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_86 /etc/fstab 2026-02-22T23:27:51.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_87 /etc/fstab 2026-02-22T23:27:51.615 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.615 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_88 /etc/fstab 2026-02-22T23:27:51.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_89 /etc/fstab 2026-02-22T23:27:51.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_90 /etc/fstab 2026-02-22T23:27:51.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_91 /etc/fstab 2026-02-22T23:27:51.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_92 /etc/fstab 2026-02-22T23:27:51.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_93 /etc/fstab 2026-02-22T23:27:51.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_94 /etc/fstab 2026-02-22T23:27:51.809 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.809 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_95 /etc/fstab 2026-02-22T23:27:51.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_96 /etc/fstab 2026-02-22T23:27:51.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_97 /etc/fstab 2026-02-22T23:27:51.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_98 /etc/fstab 2026-02-22T23:27:51.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_99 /etc/fstab 2026-02-22T23:27:51.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:686: TEST_divergent_3: for i in $(seq 1 $testobjects) 2026-02-22T23:27:51.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:688: TEST_divergent_3: rados -p test put existing_100 /etc/fstab 2026-02-22T23:27:51.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:691: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:27:51.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:691: TEST_divergent_3: wait_for_clean 2026-02-22T23:27:51.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:27:51.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:27:51.970 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:27:51.970 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:27:51.970 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:27:51.971 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:27:51.971 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:27:51.971 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:27:51.971 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:27:52.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:27:52.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:27:52.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:27:52.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:27:52.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:27:52.042 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:27:52.242 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:27:52.243 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:27:52.243 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:27:52.243 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:27:52.243 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:52.243 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:27:52.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836492 2026-02-22T23:27:52.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836492 2026-02-22T23:27:52.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492' 2026-02-22T23:27:52.323 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:52.323 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:27:52.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-22T23:27:52.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-22T23:27:52.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674' 2026-02-22T23:27:52.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:27:52.400 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:27:52.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542153 2026-02-22T23:27:52.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542153 2026-02-22T23:27:52.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836492 1-38654705674 2-60129542153' 2026-02-22T23:27:52.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:52.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836492 2026-02-22T23:27:52.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:52.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:27:52.484 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836492 2026-02-22T23:27:52.484 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:52.485 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836492 2026-02-22T23:27:52.485 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-22T23:27:52.486 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836492 2026-02-22T23:27:52.486 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:27:52.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-22T23:27:52.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:27:53.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:27:53.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:27:53.898 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-22T23:27:53.898 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:53.899 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-22T23:27:53.899 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:53.900 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:27:53.901 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-22T23:27:53.901 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:53.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-22T23:27:53.902 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705674 2026-02-22T23:27:53.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-22T23:27:53.903 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:27:54.112 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705675 -lt 38654705674 2026-02-22T23:27:54.112 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:27:54.113 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542153 2026-02-22T23:27:54.113 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:27:54.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:27:54.114 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542153 2026-02-22T23:27:54.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:27:54.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542153 2026-02-22T23:27:54.116 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542153 2026-02-22T23:27:54.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542153' 2026-02-22T23:27:54.117 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:27:54.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542153 -lt 60129542153 2026-02-22T23:27:54.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:27:54.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:27:54.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:27:54.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:27:54.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:27:54.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:27:54.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:27:54.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:27:54.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:27:54.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:27:54.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:27:54.783 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:27:54.784 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:27:54.784 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:27:54.784 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:27:55.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:27:55.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:27:55.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:27:55.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: get_pg test existing_1 2026-02-22T23:27:55.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-22T23:27:55.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-22T23:27:55.056 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-22T23:27:55.056 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-22T23:27:55.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:693: TEST_divergent_3: local pgid=1.0 2026-02-22T23:27:55.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:696: TEST_divergent_3: echo 'blackholing osds 0 2' 2026-02-22T23:27:55.258 INFO:tasks.workunit.client.0.trial122.stdout:blackholing osds 0 2 2026-02-22T23:27:55.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:697: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:27:55.449 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:27:55.449 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-22T23:27:44.137643+0000 30'100 30:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:16:26.397145+0000 0 0 2026-02-22T23:27:55.449 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:27:55.449 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:27:55.449 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:27:55.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:27:55.460 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: get_asok_path osd.0 2026-02-22T23:27:55.460 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:27:55.460 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:27:55.461 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:27:55.461 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:55.461 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:55.461 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:27:55.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: CEPH_ARGS= 2026-02-22T23:27:55.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-22T23:27:55.523 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:27:55.523 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:27:55.523 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:27:55.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:698: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:27:55.530 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: get_asok_path osd.2 2026-02-22T23:27:55.530 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:27:55.530 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:27:55.530 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:27:55.530 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:27:55.530 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:27:55.531 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:27:55.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: CEPH_ARGS= 2026-02-22T23:27:55.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:700: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-22T23:27:55.580 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:27:55.580 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:27:55.580 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:27:55.586 INFO:tasks.workunit.client.0.trial122.stdout:create new divergent objects 2026-02-22T23:27:55.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:704: TEST_divergent_3: echo 'create new divergent objects' 2026-02-22T23:27:55.586 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: seq 1 2 2026-02-22T23:27:55.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:27:55.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:705: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:27:55.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: rados -p test create newobject_1 2026-02-22T23:27:55.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:710: TEST_divergent_3: echo 'writing divergent objects' 2026-02-22T23:27:55.588 INFO:tasks.workunit.client.0.trial122.stdout:writing divergent objects 2026-02-22T23:27:55.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:707: TEST_divergent_3: rados -p test create newobject_2 2026-02-22T23:27:55.589 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: seq 1 2 2026-02-22T23:27:55.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-22T23:27:55.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:711: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_WRITE) 2026-02-22T23:27:55.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: rados -p test put existing_1 /etc/resolv.conf 2026-02-22T23:27:55.590 INFO:tasks.workunit.client.0.trial122.stdout:remove divergent objects 2026-02-22T23:27:55.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:716: TEST_divergent_3: echo 'remove divergent objects' 2026-02-22T23:27:55.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:713: TEST_divergent_3: rados -p test put existing_2 /etc/resolv.conf 2026-02-22T23:27:55.591 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: seq 1 2 2026-02-22T23:27:55.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-22T23:27:55.592 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: expr 1 + 2 2026-02-22T23:27:55.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: rmi=3 2026-02-22T23:27:55.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:717: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_REMOVE) 2026-02-22T23:27:55.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: rados -p test rm existing_3 2026-02-22T23:27:55.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: expr 2 + 2 2026-02-22T23:27:55.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:719: TEST_divergent_3: rmi=4 2026-02-22T23:27:55.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:722: TEST_divergent_3: sleep 10 2026-02-22T23:27:55.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:720: TEST_divergent_3: rados -p test rm existing_4 2026-02-22T23:28:05.597 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:723: TEST_divergent_3: killall -9 rados 2026-02-22T23:28:05.641 INFO:tasks.workunit.client.0.trial122.stderr:rados: no process found 2026-02-22T23:28:05.641 INFO:tasks.workunit.client.0.trial122.stdout:killing all the osds 2026-02-22T23:28:05.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:726: TEST_divergent_3: echo 'killing all the osds' 2026-02-22T23:28:05.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:727: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:28:05.841 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:28:05.841 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 active+clean 2026-02-22T23:27:44.137643+0000 30'106 30:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:16:26.397145+0000 0 0 2026-02-22T23:28:05.841 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:28:05.841 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:28:05.841 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:28:05.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:728: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd 2026-02-22T23:28:05.851 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:28:05.851 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:28:05.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:28:05.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:28:05.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:28:05.960 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:28:05.960 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-22T23:28:05.960 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.0 2026-02-22T23:28:06.236 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:28:06.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-22T23:28:06.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.1 2026-02-22T23:28:06.456 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:28:06.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:729: TEST_divergent_3: for i in $osds 2026-02-22T23:28:06.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:731: TEST_divergent_3: ceph osd down osd.2 2026-02-22T23:28:06.718 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:28:06.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:28:06.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: ceph osd out osd.0 2026-02-22T23:28:06.978 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already out. 2026-02-22T23:28:06.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:733: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:28:06.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:735: TEST_divergent_3: ceph osd out osd.2 2026-02-22T23:28:07.255 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already out. 2026-02-22T23:28:07.264 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:739: TEST_divergent_3: echo 'bringing up non_divergent 0 2' 2026-02-22T23:28:07.265 INFO:tasks.workunit.client.0.trial122.stdout:bringing up non_divergent 0 2 2026-02-22T23:28:07.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:740: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:28:07.468 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:28:07.469 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 71822 0 0 16 90 16 stale+active+clean 2026-02-22T23:27:44.137643+0000 30'106 30:129 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:16:26.397145+0000 0 0 2026-02-22T23:28:07.469 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:28:07.469 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:28:07.469 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: activate_osd td/divergent-priors 0 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:28:07.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:28:07.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:28:07.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:28:07.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:28:07.482 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:28:07.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:28:07.483 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-22T23:28:07.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:28:07.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:28:07.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:28:07.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:28:07.500 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:07.498+0000 7fdf97a908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:28:07.501 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:07.498+0000 7fdf97a908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:28:07.502 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:07.498+0000 7fdf97a908c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:28:07.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:28:07.682 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:28:07.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:07.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:28:07.880 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:08.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:08.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:08.882 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:28:08.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:28:08.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:08.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:28:08.992 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:08.990+0000 7fdf97a908c0 -1 Falling back to public interface 2026-02-22T23:28:09.089 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:10.090 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:10.090 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:10.090 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:28:10.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:28:10.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:10.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:28:10.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:11.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:11.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:11.298 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:28:11.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:28:11.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:11.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:28:11.501 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:12.253 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:12.250+0000 7fdf97a908c0 -1 osd.0 30 log_to_monitors true 2026-02-22T23:28:12.503 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:12.503 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:12.503 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:28:12.504 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:28:12.504 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:12.504 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:28:12.706 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up out weight 0 up_from 37 up_thru 0 down_at 31 last_clean_interval [5,30) [v2:127.0.0.1:6802/994462284,v1:127.0.0.1:6803/994462284] [v2:127.0.0.1:6804/994462284,v1:127.0.0.1:6805/994462284] exists,up a9320b9d-2589-436d-a8bb-32a33f236ef9 2026-02-22T23:28:12.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:28:12.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:28:12.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:28:12.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:741: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:743: TEST_divergent_3: activate_osd td/divergent-priors 2 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:28:12.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:28:12.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:28:12.712 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:28:12.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:28:12.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:28:12.714 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-22T23:28:12.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:28:12.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:28:12.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:28:12.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:28:12.730 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:12.726+0000 7f1670f6b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:28:12.731 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:12.730+0000 7f1670f6b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:28:12.732 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:12.730+0000 7f1670f6b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:28:12.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:28:12.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:28:12.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:28:12.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:28:12.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:28:12.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:12.907 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:28:12.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:28:12.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:12.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:28:13.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:14.001 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:13.998+0000 7f1670f6b8c0 -1 Falling back to public interface 2026-02-22T23:28:14.112 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:14.112 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:14.113 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:28:14.113 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:28:14.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:14.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:28:14.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:15.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:15.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:15.311 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:28:15.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:28:15.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:15.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:28:15.507 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:16.509 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:16.509 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:16.510 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:28:16.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:28:16.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:16.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:28:16.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:28:17.066 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:28:17.062+0000 7f1670f6b8c0 -1 osd.2 30 log_to_monitors true 2026-02-22T23:28:17.715 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:28:17.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:28:17.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:28:17.716 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:28:17.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:28:17.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:28:17.915 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up out weight 0 up_from 40 up_thru 0 down_at 33 last_clean_interval [14,30) [v2:127.0.0.1:6810/3891353798,v1:127.0.0.1:6811/3891353798] [v2:127.0.0.1:6812/3891353798,v1:127.0.0.1:6813/3891353798] exists,up ddddd36a-efe9-46bc-8e09-84f6fdabe0bc 2026-02-22T23:28:17.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:28:17.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:28:17.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:28:17.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:28:17.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph osd in osd.0 2026-02-22T23:28:18.163 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already in. 2026-02-22T23:28:18.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:745: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:28:18.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:747: TEST_divergent_3: ceph osd in osd.2 2026-02-22T23:28:18.468 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already in. 2026-02-22T23:28:18.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:28:18.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:750: TEST_divergent_3: wait_for_clean 2026-02-22T23:28:18.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:28:18.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:28:18.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:28:18.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:28:18.479 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:28:18.479 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:28:18.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:28:18.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:28:18.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:28:18.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:28:18.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:28:18.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:28:18.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:28:18.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:28:18.550 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:28:18.746 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:28:18.746 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:28:18.747 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:28:18.747 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:28:18.747 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:28:18.747 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:28:18.825 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789956 2026-02-22T23:28:18.826 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789956 2026-02-22T23:28:18.826 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-158913789956' 2026-02-22T23:28:18.826 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:28:18.826 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:28:18.886 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:28:18.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:28:18.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:28:18.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:28:18.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:28:18.890 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:28:18.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691843 2026-02-22T23:28:18.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691843 2026-02-22T23:28:18.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-158913789956 2-171798691843' 2026-02-22T23:28:18.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:28:18.965 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-158913789956 2026-02-22T23:28:18.965 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:28:18.966 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:28:18.967 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-158913789956 2026-02-22T23:28:18.967 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:28:18.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789956 2026-02-22T23:28:18.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 158913789956' 2026-02-22T23:28:18.969 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 158913789956 2026-02-22T23:28:18.969 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:28:19.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789953 -lt 158913789956 2026-02-22T23:28:19.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:28:20.185 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:28:20.186 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:28:20.383 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789956 -lt 158913789956 2026-02-22T23:28:20.383 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:28:20.383 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-171798691843 2026-02-22T23:28:20.384 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:28:20.385 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:28:20.385 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-171798691843 2026-02-22T23:28:20.385 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:28:20.387 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691843 2026-02-22T23:28:20.387 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 171798691843 2026-02-22T23:28:20.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 171798691843' 2026-02-22T23:28:20.388 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:28:20.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691843 -lt 171798691843 2026-02-22T23:28:20.591 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:28:20.592 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:20.592 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:20.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:28:20.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:20.857 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:20.857 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:20.857 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:20.857 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:20.858 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:20.858 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:21.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:21.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:21.056 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:21.056 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:21.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:28:21.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:28:21.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:28:21.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:28:21.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:21.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:28:21.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:21.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:21.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:21.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:21.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:21.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:21.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:21.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:21.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:21.621 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:21.621 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:21.622 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:21.884 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:21.884 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:21.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:21.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:21.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:21.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:21.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:21.886 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:21.886 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:22.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:22.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:22.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:28:22.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:22.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:28:22.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:22.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:22.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:22.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:22.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:22.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:22.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:22.347 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:22.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:22.545 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:22.545 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:22.545 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:22.832 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:22.833 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:22.833 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:23.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:23.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:23.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:28:23.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:23.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:28:23.498 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:23.499 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:23.500 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:23.500 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:23.500 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:23.500 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:23.500 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:23.500 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:23.704 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:23.704 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:23.705 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:23.705 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:23.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:23.975 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:24.240 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:24.240 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:24.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:28:24.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:24.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:28:25.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:25.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:25.042 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:25.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:25.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:25.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:25.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:25.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:25.242 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:25.243 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:25.243 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:25.243 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:25.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:25.511 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:25.511 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:25.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:25.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:25.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:28:25.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:25.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:28:27.382 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:27.382 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:27.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:27.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:27.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:27.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:27.383 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:27.383 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:27.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:27.588 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:27.588 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:27.588 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:27.854 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:27.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:27.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:27.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:27.855 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:27.855 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:28.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:28.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:28.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:28:28.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:28.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:28:31.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:31.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:31.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:31.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:31.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:31.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:31.320 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:31.321 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:31.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:31.524 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:31.524 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:31.525 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:31.781 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:31.782 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:31.782 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:32.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:32.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:32.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:28:32.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:32.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:28:38.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:38.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:38.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:38.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:38.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:38.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:38.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:38.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:38.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:38.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:38.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:38.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:38.903 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:38.904 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:39.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:39.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:39.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:28:39.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:39.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:28:46.456 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:46.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:46.458 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:46.458 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:46.458 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:46.458 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:46.458 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:46.458 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:46.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:46.658 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:46.658 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:46.658 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:46.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:46.924 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:46.924 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:47.178 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:47.179 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:47.179 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:28:47.179 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:28:47.430 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:28:47.430 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "2fe7ed03-3179-401e-b371-2a8840fa84dc", 2026-02-22T23:28:47.430 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:28:47.430 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:28:47.430 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:28:47.429832+0000", 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.431 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:28:47.432 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "count": 32 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.433 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.434 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.435 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.436 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.437 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.438 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.439 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.440 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:28:47.441 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:28:47.442 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:27:14.744609Z", 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:27:14.744609Z", 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:28:47.443 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:28:47.444 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:28:47.445 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:28:47.446 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 47, 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:27:14.820947+0000", 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:28:19.447775+0000", 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:28:17.221488+0000", 2026-02-22T23:28:47.447 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:28:18.413811+0000", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 9, 2026-02-22T23:28:47.448 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.449 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:27:42.738487+0000", 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 1, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool", 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 0, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:28:47.450 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "on", 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 32, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 32, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 32, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 32, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:28:47.451 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "46", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "46", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "selfmanaged", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 0, 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 0, 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [], 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:28:47.452 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:28:47.453 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "", 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:28:47.454 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 0, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {}, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "read_balance": { 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "score_type": "Fair distribution", 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "score_acting": 2.9900000095367432, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "score_stable": 2.9900000095367432, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_score": 0.67000001668930054, 2026-02-22T23:28:47.455 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_acting": 2, 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_stable": 2, 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity_weighted": 1, 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity": 1, 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity_weighted": 1 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "a9320b9d-2589-436d-a8bb-32a33f236ef9", 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:28:47.456 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 5, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 30, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 37, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 46, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 31, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.457 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.458 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:28:47.459 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.460 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.461 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/994462284", 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/994462284", 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/994462284", 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/994462284", 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.462 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "cc902fd6-3159-44fe-844d-063a2679a371", 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 22, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 32, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:28:47.463 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.464 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:28:47.465 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:28:47.466 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/1381058531", 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/1381058531", 2026-02-22T23:28:47.467 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1381058531", 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1381058531", 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "ddddd36a-efe9-46bc-8e09-84f6fdabe0bc", 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 14, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 30, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 40, 2026-02-22T23:28:47.468 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 33, 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.469 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.470 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:28:47.471 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3891353798", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3891353798", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3891353798", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3891353798", 2026-02-22T23:28:47.472 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:28:06.182061+0000", 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:27:23.863828+0000", 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:28:47.473 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:28:06.406367+0000", 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:27:32.815818+0000", 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:28:06.666255+0000", 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:28:47.474 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:27:41.030720+0000", 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:28:47.475 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.476 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/994462284,v1:127.0.0.1:6805/994462284]", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:28:47.477 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:27:17.252338Z", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/994462284,v1:127.0.0.1:6803/994462284]", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/994462284,v1:127.0.0.1:6809/994462284]", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/994462284,v1:127.0.0.1:6807/994462284]", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:28:47.478 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/1381058531,v1:127.0.0.1:6813/1381058531]", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:28:47.479 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:28:47.480 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:27:26.465719Z", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/1381058531,v1:127.0.0.1:6811/1381058531]", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1381058531,v1:127.0.0.1:6817/1381058531]", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1381058531,v1:127.0.0.1:6815/1381058531]", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:28:47.481 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3891353798,v1:127.0.0.1:6813/3891353798]", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:28:47.482 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:27:34.460263Z", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3891353798,v1:127.0.0.1:6811/3891353798]", 2026-02-22T23:28:47.483 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3891353798,v1:127.0.0.1:6817/3891353798]", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3891353798,v1:127.0.0.1:6815/3891353798]", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 0, 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:28:47.484 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [] 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 37 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 41 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 47, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.485 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:28:47.486 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:28:47.487 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stderr:report 2886206479 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.488 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:28:47.489 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.490 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.491 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:28:47.492 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:28:47.493 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 0, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:27:14:820728+0000", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:28:47.494 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 32, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 32, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:28:47.495 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 72800, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 100, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 100, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:28:47.496 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 512, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 512, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "up": 64, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 64, 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:28:47.497 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 65, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 84036, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2208, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488764, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036494336, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2260992, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 534673, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679083 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:28:47.498 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 65, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 84036, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2208, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488764, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036494336, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:28:47.499 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2260992, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 534673, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679083 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 32, 2026-02-22T23:28:47.500 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 72800, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 100, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 100, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:28:47.501 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1220608, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 217422, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 512, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 512, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "up": 64, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 64, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:28:47.502 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 158913789961 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705677 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 171798691848 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num": 32 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 32, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 32, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 32, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:28:47.503 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 4, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:28:33.297400+0000", 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "services": { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "osd": { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "daemons": { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "summary": "", 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "0": { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "2": { 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.504 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 44, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 229, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 330, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:753: TEST_divergent_3: expr 2 + 2 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:753: TEST_divergent_3: objname=existing_4 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:754: TEST_divergent_3: echo 'writing non-divergent object existing_4' 2026-02-22T23:28:47.505 INFO:tasks.workunit.client.0.trial122.stdout:writing non-divergent object existing_4 2026-02-22T23:28:47.506 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:755: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:28:47.627 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:28:47.627 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194787+0000 0 0 2026-02-22T23:28:47.627 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194793+0000 0 0 2026-02-22T23:28:47.627 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194677+0000 0 0 2026-02-22T23:28:47.627 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194748+0000 0 0 2026-02-22T23:28:47.627 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194615+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194653+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194607+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194815+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194778+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194720+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194763+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194598+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194623+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194727+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194638+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194670+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 47:141 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:36.212750+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194713+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194808+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194825+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194661+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194691+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194698+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194685+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194631+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194645+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194771+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194801+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194706+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194741+0000 0 0 2026-02-22T23:28:47.628 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194756+0000 0 0 2026-02-22T23:28:47.629 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194735+0000 0 0 2026-02-22T23:28:47.629 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:28:47.629 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:28:47.629 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:28:47.636 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:756: TEST_divergent_3: rados -p test put existing_4 /etc/resolv.conf 2026-02-22T23:28:47.666 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:28:47.666 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:758: TEST_divergent_3: wait_for_clean 2026-02-22T23:28:47.666 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:28:47.666 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:28:47.666 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:28:47.667 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:28:47.667 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:28:47.667 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:28:47.667 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:28:47.667 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:28:47.667 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:28:47.738 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:28:47.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:28:47.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:28:47.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:28:47.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:28:47.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:28:47.943 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:28:47.944 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:28:47.944 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:28:47.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:28:47.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:28:47.944 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:28:48.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=158913789963 2026-02-22T23:28:48.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 158913789963 2026-02-22T23:28:48.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-158913789963' 2026-02-22T23:28:48.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:28:48.028 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:28:48.086 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:28:48.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:28:48.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:28:48.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:28:48.088 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:28:48.088 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:28:48.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=171798691851 2026-02-22T23:28:48.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 171798691851 2026-02-22T23:28:48.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-158913789963 2-171798691851' 2026-02-22T23:28:48.164 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:28:48.165 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-158913789963 2026-02-22T23:28:48.165 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:28:48.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:28:48.167 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-158913789963 2026-02-22T23:28:48.167 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:28:48.169 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=158913789963 2026-02-22T23:28:48.169 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 158913789963' 2026-02-22T23:28:48.169 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 158913789963 2026-02-22T23:28:48.169 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:28:48.381 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789961 -lt 158913789963 2026-02-22T23:28:48.381 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:28:49.382 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:28:49.383 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:28:49.579 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 158913789964 -lt 158913789963 2026-02-22T23:28:49.579 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:28:49.580 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-171798691851 2026-02-22T23:28:49.580 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:28:49.581 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:28:49.582 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-171798691851 2026-02-22T23:28:49.582 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:28:49.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=171798691851 2026-02-22T23:28:49.584 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 171798691851' 2026-02-22T23:28:49.584 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 171798691851 2026-02-22T23:28:49.584 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:28:49.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 171798691851 -lt 171798691851 2026-02-22T23:28:49.797 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:28:49.797 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:49.797 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:50.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 32 == 0 2026-02-22T23:28:50.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:50.064 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:50.064 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:50.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:50.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:50.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:50.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:50.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:50.265 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:50.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:50.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:50.522 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:50.522 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:28:50.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:28:50.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:28:50.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:50.523 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:28:50.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:50.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:50.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:50.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:50.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:50.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:50.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:50.625 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:50.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:50.829 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:50.830 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:50.830 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:51.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:51.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:51.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:51.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:51.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:51.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:51.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:51.094 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:51.094 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:51.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:51.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:51.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:28:51.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:51.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:28:51.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:51.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:51.561 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:51.561 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:51.561 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:51.561 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:51.562 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:51.562 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:51.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:51.763 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:51.764 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:51.764 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:52.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:52.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:52.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:52.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:52.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:52.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:52.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:52.023 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:52.023 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:52.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:52.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:52.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:28:52.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:52.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:28:52.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:52.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:52.687 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:52.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:52.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:52.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:52.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:52.688 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:52.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:52.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:52.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:52.893 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:53.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:53.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:53.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:53.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:53.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:53.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:53.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:53.176 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:53.177 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:53.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:53.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:53.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:28:53.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:53.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:28:54.238 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:54.238 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:54.239 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:54.239 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:54.239 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:54.240 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:54.240 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:54.240 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:54.442 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:54.442 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:54.443 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:54.443 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:54.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:54.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:54.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:54.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:54.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:54.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:54.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:54.698 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:54.698 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:54.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:54.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:54.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:28:54.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:54.954 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:28:56.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:28:56.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:28:56.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:28:56.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:28:56.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:28:56.556 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:28:56.556 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:28:56.556 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:28:56.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:28:56.752 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:28:56.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:28:56.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:28:57.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:28:57.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:28:57.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:28:57.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:28:57.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:28:57.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:28:57.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:28:57.009 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:28:57.009 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:28:57.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:28:57.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:28:57.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:28:57.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:28:57.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:29:00.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:29:00.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:29:00.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:29:00.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:29:00.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:29:00.463 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:29:00.463 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:29:00.463 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:29:00.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:29:00.660 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:29:00.661 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:29:00.661 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:29:00.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:29:00.925 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:29:01.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:29:01.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:29:01.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:29:01.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:29:01.188 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:29:07.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:29:07.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:29:07.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:29:07.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:29:07.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:29:07.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:29:07.590 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:29:07.591 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:29:07.794 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:29:07.794 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:29:07.795 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:29:07.795 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:29:08.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:29:08.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:29:08.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:29:08.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:29:08.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:29:08.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:29:08.055 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:29:08.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:29:08.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:29:08.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:29:08.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:29:08.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:29:08.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:29:08.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:29:15.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:29:15.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:29:15.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:29:15.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:29:15.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:29:15.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:29:15.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:29:15.620 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:29:15.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:29:15.821 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:29:15.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:29:15.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 32 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:29:16.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:29:16.094 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:29:16.094 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:29:16.355 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:29:16.355 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:29:16.355 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:29:16.355 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:29:16.614 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:29:16.614 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "2fe7ed03-3179-401e-b371-2a8840fa84dc", 2026-02-22T23:29:16.614 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:29:16.614230+0000", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:29:16.615 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.616 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 100/300 objects degraded (33.333%), 32 pgs degraded", 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "count": 32 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:29:16.617 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.2 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.3 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.618 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.4 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.5 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.6 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.7 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.619 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.8 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.9 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.a is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.b is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.c is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.d is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.e is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.f is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.10 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.621 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.11 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.12 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.13 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.14 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.622 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.15 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.16 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.17 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.18 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.623 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.19 is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1a is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1b is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1c is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.624 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1d is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1e is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.1f is active+undersized+degraded, acting [0,2]" 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:29:16.625 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.626 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:27:14.744609Z", 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:27:14.744609Z", 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:29:16.627 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:29:16.628 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:29:16.629 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:29:16.630 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 47, 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:27:14.820947+0000", 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:28:19.447775+0000", 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:28:17.221488+0000", 2026-02-22T23:29:16.631 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:28:18.413811+0000", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 9, 2026-02-22T23:29:16.632 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:29:16.633 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:27:42.738487+0000", 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 1, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool", 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 0, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:29:16.634 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "on", 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 32, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 32, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 32, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 32, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:29:16.635 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "46", 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "46", 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "selfmanaged", 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 0, 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 0, 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [], 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:29:16.636 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:29:16.637 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "", 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 0, 2026-02-22T23:29:16.638 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {}, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "read_balance": { 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "score_type": "Fair distribution", 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "score_acting": 2.9900000095367432, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "score_stable": 2.9900000095367432, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_score": 0.67000001668930054, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_acting": 2, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "raw_score_stable": 2, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity_weighted": 1, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity": 1, 2026-02-22T23:29:16.639 INFO:tasks.workunit.client.0.trial122.stdout: "average_primary_affinity_weighted": 1 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "a9320b9d-2589-436d-a8bb-32a33f236ef9", 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 5, 2026-02-22T23:29:16.640 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 30, 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 37, 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 46, 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 31, 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.641 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.642 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.643 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 994462284 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/994462284", 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/994462284", 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/994462284", 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/994462284", 2026-02-22T23:29:16.645 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "cc902fd6-3159-44fe-844d-063a2679a371", 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:29:16.646 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 22, 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 32, 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.647 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.648 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.649 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.650 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1381058531 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/1381058531", 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/1381058531", 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/1381058531", 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/1381058531", 2026-02-22T23:29:16.651 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "ddddd36a-efe9-46bc-8e09-84f6fdabe0bc", 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 14, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 30, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 40, 2026-02-22T23:29:16.652 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 33, 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.653 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.654 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.655 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3891353798 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3891353798", 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3891353798", 2026-02-22T23:29:16.656 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3891353798", 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3891353798", 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:28:06.182061+0000", 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:29:16.657 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:27:23.863828+0000", 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:28:06.406367+0000", 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:27:32.815818+0000", 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:29:16.658 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:28:06.666255+0000", 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:27:41.030720+0000", 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:29:16.659 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.660 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/994462284,v1:127.0.0.1:6805/994462284]", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:29:16.661 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:27:17.252338Z", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:29:16.662 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/994462284,v1:127.0.0.1:6803/994462284]", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/994462284,v1:127.0.0.1:6809/994462284]", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/994462284,v1:127.0.0.1:6807/994462284]", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:29:16.663 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/1381058531,v1:127.0.0.1:6813/1381058531]", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:29:16.664 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:27:26.465719Z", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/1381058531,v1:127.0.0.1:6811/1381058531]", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/1381058531,v1:127.0.0.1:6817/1381058531]", 2026-02-22T23:29:16.665 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/1381058531,v1:127.0.0.1:6815/1381058531]", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3891353798,v1:127.0.0.1:6813/3891353798]", 2026-02-22T23:29:16.666 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:29:16.667 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:27:34.460263Z", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3891353798,v1:127.0.0.1:6811/3891353798]", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3891353798,v1:127.0.0.1:6817/3891353798]", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3891353798,v1:127.0.0.1:6815/3891353798]", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:29:16.668 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 0, 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [] 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 37 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.669 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 41 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 47, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:29:16.670 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.671 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.672 INFO:tasks.workunit.client.0.trial122.stderr:report 3563035379 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:29:16.673 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.674 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.675 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.676 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:29:16.677 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 0, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:29:16.678 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:27:14:820728+0000", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:29:16.679 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 32, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 32, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73039, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:29:16.680 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:29:16.681 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 513, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 513, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "up": 64, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 64, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 65, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:29:16.682 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 84036, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2208, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488764, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036494336, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2260992, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 535151, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679083 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:29:16.683 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 65, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 84036, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2208, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488764, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036494336, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2260992, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 535151, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:29:16.684 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679083 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 32, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73039, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 100, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 0, 2026-02-22T23:29:16.685 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 300, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 100, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 100, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:29:16.686 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1220608, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 217900, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 513, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 513, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "up": 64, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 64, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 158913789969 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:29:16.687 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705677 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 171798691856 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num": 32 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 32, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 32, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 32, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.688 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 5, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:29:15.304117+0000", 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "services": { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "osd": { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "daemons": { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "summary": "", 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "2": { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 58, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 283, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 398, 2026-02-22T23:29:16.689 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:761: TEST_divergent_3: echo 'delay recovery' 2026-02-22T23:29:16.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:762: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:16.809 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:16.809 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194787+0000 0 0 2026-02-22T23:29:16.809 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194793+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194677+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194748+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194615+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194653+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194607+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194815+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194778+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194720+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194763+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194598+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194623+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194727+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194638+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194670+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 47:141 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:36.212750+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194713+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 47:141 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194825+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194661+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194691+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194698+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194685+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194631+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194645+0000 0 0 2026-02-22T23:29:16.810 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194771+0000 0 0 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194801+0000 0 0 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194706+0000 0 0 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194741+0000 0 0 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194756+0000 0 0 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194735+0000 0 0 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:16.811 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:16.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:29:16.818 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: get_asok_path osd.0 2026-02-22T23:29:16.818 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:29:16.818 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:29:16.819 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:29:16.819 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:16.819 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:16.820 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:29:16.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: CEPH_ARGS= 2026-02-22T23:29:16.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok set_recovery_delay 100000 2026-02-22T23:29:16.876 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:29:16.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:763: TEST_divergent_3: for i in $non_divergent 2026-02-22T23:29:16.883 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: get_asok_path osd.2 2026-02-22T23:29:16.883 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:29:16.883 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:29:16.883 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:29:16.884 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:16.884 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:16.884 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:29:16.884 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: CEPH_ARGS= 2026-02-22T23:29:16.884 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:765: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok set_recovery_delay 100000 2026-02-22T23:29:16.938 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:29:16.944 INFO:tasks.workunit.client.0.trial122.stdout:revive divergent 1 2026-02-22T23:29:16.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:769: TEST_divergent_3: echo 'revive divergent 1' 2026-02-22T23:29:16.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:770: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:17.147 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:17.147 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194787+0000 0 0 2026-02-22T23:29:17.147 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194793+0000 0 0 2026-02-22T23:29:17.147 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194677+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194748+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194615+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194653+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194607+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194815+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194778+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194720+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194763+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194598+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194623+0000 0 0 2026-02-22T23:29:17.148 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194727+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194638+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194670+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 47:141 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:36.212750+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194713+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 47:141 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194825+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194661+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194691+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194698+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194685+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194631+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194645+0000 0 0 2026-02-22T23:29:17.149 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194771+0000 0 0 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194801+0000 0 0 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194706+0000 0 0 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194741+0000 0 0 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194756+0000 0 0 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 47:140 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:28:25.194735+0000 0 0 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:17.150 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:17.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:771: TEST_divergent_3: ceph osd set noup 2026-02-22T23:29:17.473 INFO:tasks.workunit.client.0.trial122.stderr:noup is set 2026-02-22T23:29:17.486 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:772: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:29:17.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:17.488 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:29:17.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:29:17.490 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:29:17.490 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:29:17.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:29:17.491 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:29:17.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:29:17.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:29:17.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:29:17.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:29:17.511 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:17.505+0000 7ff1699c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:17.512 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:17.509+0000 7ff1699c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:17.513 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:17.509+0000 7ff1699c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:17.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-22T23:29:17.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:773: TEST_divergent_3: sleep 5 2026-02-22T23:29:18.752 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:18.749+0000 7ff1699c88c0 -1 Falling back to public interface 2026-02-22T23:29:21.753 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:21.749+0000 7ff1699c88c0 -1 osd.1 30 log_to_monitors true 2026-02-22T23:29:22.688 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery divergent 2026-02-22T23:29:22.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:775: TEST_divergent_3: echo 'delay recovery divergent' 2026-02-22T23:29:22.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:776: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:22.881 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:22.881 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956407+0000 0 0 2026-02-22T23:29:22.881 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956413+0000 0 0 2026-02-22T23:29:22.881 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956324+0000 0 0 2026-02-22T23:29:22.881 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956379+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956278+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956308+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956273+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956430+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956401+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956358+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956390+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956267+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956284+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956364+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956297+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956319+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956260+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956353+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956424+0000 0 0 2026-02-22T23:29:22.882 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956436+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956314+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956335+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956342+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956330+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956290+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956302+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956396+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956418+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956348+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956374+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956385+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956369+0000 0 0 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:22.883 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:22.889 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: get_asok_path osd.1 2026-02-22T23:29:22.889 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:29:22.890 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:29:22.890 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:29:22.890 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:22.890 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:22.890 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:29:22.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: CEPH_ARGS= 2026-02-22T23:29:22.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:777: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok set_recovery_delay 100000 2026-02-22T23:29:22.946 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:29:22.951 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:779: TEST_divergent_3: ceph osd unset noup 2026-02-22T23:29:23.291 INFO:tasks.workunit.client.0.trial122.stderr:noup is unset 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:781: TEST_divergent_3: wait_for_osd up 0 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:29:23.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:29:23.304 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:29:23.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:29:23.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:29:23.510 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 37 up_thru 46 down_at 31 last_clean_interval [5,30) [v2:127.0.0.1:6802/994462284,v1:127.0.0.1:6803/994462284] [v2:127.0.0.1:6804/994462284,v1:127.0.0.1:6805/994462284] exists,up a9320b9d-2589-436d-a8bb-32a33f236ef9 2026-02-22T23:29:23.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:782: TEST_divergent_3: wait_for_osd up 1 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:29:23.512 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:29:23.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:29:23.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:29:23.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:29:23.715 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 52 up_thru 52 down_at 32 last_clean_interval [9,30) [v2:127.0.0.1:6818/327770727,v1:127.0.0.1:6819/327770727] [v2:127.0.0.1:6820/327770727,v1:127.0.0.1:6821/327770727] exists,up cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:783: TEST_divergent_3: wait_for_osd up 2 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:29:23.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:29:23.718 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:29:23.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:29:23.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:29:23.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:29:23.914 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 40 up_thru 0 down_at 33 last_clean_interval [14,30) [v2:127.0.0.1:6810/3891353798,v1:127.0.0.1:6811/3891353798] [v2:127.0.0.1:6812/3891353798,v1:127.0.0.1:6813/3891353798] exists,up ddddd36a-efe9-46bc-8e09-84f6fdabe0bc 2026-02-22T23:29:23.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:29:23.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:29:23.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:29:23.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:785: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:24.106 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956407+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956413+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956324+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956379+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956278+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956308+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956273+0000 0 0 2026-02-22T23:29:24.107 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956430+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956401+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956358+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956390+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956267+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956284+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956364+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956297+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956319+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956260+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956353+0000 0 0 2026-02-22T23:29:24.108 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956424+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956436+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956314+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956335+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956342+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956330+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956290+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956302+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956396+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956418+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956348+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956374+0000 0 0 2026-02-22T23:29:24.109 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956385+0000 0 0 2026-02-22T23:29:24.110 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956369+0000 0 0 2026-02-22T23:29:24.110 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:24.110 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:24.110 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:24.115 INFO:tasks.workunit.client.0.trial122.stdout:wait for peering 2026-02-22T23:29:24.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:786: TEST_divergent_3: echo 'wait for peering' 2026-02-22T23:29:24.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:787: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:24.320 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:24.320 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956407+0000 0 0 2026-02-22T23:29:24.320 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956413+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956324+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956379+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956278+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956308+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956273+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956430+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956401+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956358+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956390+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956267+0000 0 0 2026-02-22T23:29:24.321 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956284+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956364+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956297+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956319+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956260+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956353+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956424+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956436+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956314+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956335+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956342+0000 0 0 2026-02-22T23:29:24.322 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956330+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956290+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956302+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956396+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956418+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956348+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956374+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956385+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956369+0000 0 0 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:24.323 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:24.324 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:24.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:788: TEST_divergent_3: rados -p test put foo /etc/fstab 2026-02-22T23:29:24.358 INFO:tasks.workunit.client.0.trial122.stdout:killing divergent 1 2026-02-22T23:29:24.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:792: TEST_divergent_3: echo 'killing divergent 1' 2026-02-22T23:29:24.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:793: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:24.554 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956407+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956413+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956324+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956379+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956278+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956308+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956273+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956430+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956401+0000 0 0 2026-02-22T23:29:24.555 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956358+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956390+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956267+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956284+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956364+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956297+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956319+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956260+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956353+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956424+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956436+0000 0 0 2026-02-22T23:29:24.556 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956314+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956335+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956342+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956330+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956290+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956302+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956396+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956418+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956348+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956374+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956385+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956369+0000 0 0 2026-02-22T23:29:24.557 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:24.558 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:24.558 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:24.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:794: TEST_divergent_3: kill_daemons td/divergent-priors KILL osd.1 2026-02-22T23:29:24.564 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:29:24.564 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:29:24.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:29:24.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:29:24.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:29:24.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:29:24.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:797: TEST_divergent_3: expfile=td/divergent-priors/exp.20359.out 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:798: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20359.out 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-22T23:29:24.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op export-remove --pgid 1.0 --file td/divergent-priors/exp.20359.out 2026-02-22T23:29:25.314 INFO:tasks.workunit.client.0.trial122.stderr:Exporting 1.0 info 1.0( v 30'100 (30'90,30'100] local-lis/les=52/53 n=3 ec=22/22 lis/c=52/52 les/c/f=53/53/0 sis=52) 2026-02-22T23:29:25.316 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:023dd3d2:::existing_12:head# 2026-02-22T23:29:25.316 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:04c40a0c:::existing_93:head# 2026-02-22T23:29:25.316 INFO:tasks.workunit.client.0.trial122.stderr:Read #1:064ff5fb:::existing_36:head# 2026-02-22T23:29:25.316 INFO:tasks.workunit.client.0.trial122.stderr:Export successful 2026-02-22T23:29:25.317 INFO:tasks.workunit.client.0.trial122.stdout: marking collection for removal 2026-02-22T23:29:25.317 INFO:tasks.workunit.client.0.trial122.stdout:setting '_remove' omap key 2026-02-22T23:29:25.317 INFO:tasks.workunit.client.0.trial122.stdout:finish_remove_pgs 1.0_head removing 1.0 2026-02-22T23:29:25.317 INFO:tasks.workunit.client.0.trial122.stderr:Remove successful 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:799: TEST_divergent_3: _objectstore_tool_nodown td/divergent-priors 1 --op import --file td/divergent-priors/exp.20359.out 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-22T23:29:25.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op import --file td/divergent-priors/exp.20359.out 2026-02-22T23:29:26.501 INFO:tasks.workunit.client.0.trial122.stdout:get_pg_num_history pg_num_history pg_num_history(e53 pg_nums {1={22=1,46=32}} deleted_pools ) 2026-02-22T23:29:26.501 INFO:tasks.workunit.client.0.trial122.stdout:Importing pgid 1.0 2026-02-22T23:29:26.501 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:023dd3d2:::existing_12:head# 2026-02-22T23:29:26.501 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:29:26.501 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:04c40a0c:::existing_93:head# 2026-02-22T23:29:26.502 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:29:26.502 INFO:tasks.workunit.client.0.trial122.stdout:Write #1:064ff5fb:::existing_36:head# 2026-02-22T23:29:26.502 INFO:tasks.workunit.client.0.trial122.stdout:snapset 0=[]:{} 2026-02-22T23:29:26.503 INFO:tasks.workunit.client.0.trial122.stdout:write_pg epoch 53 info 1.0( v 30'100 (30'90,30'100] local-lis/les=52/53 n=3 ec=22/22 lis/c=52/52 les/c/f=53/53/0 sis=52) 2026-02-22T23:29:26.503 INFO:tasks.workunit.client.0.trial122.stdout:Import successful 2026-02-22T23:29:27.036 INFO:tasks.workunit.client.0.trial122.stdout:reviving divergent 1 2026-02-22T23:29:27.037 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:801: TEST_divergent_3: echo 'reviving divergent 1' 2026-02-22T23:29:27.037 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:802: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463304+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956407+0000 0 0 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463997+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956413+0000 0 0 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461154+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956324+0000 0 0 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463441+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956379+0000 0 0 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460798+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956278+0000 0 0 2026-02-22T23:29:27.221 INFO:tasks.workunit.client.0.trial122.stdout:1.9 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460955+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956308+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460666+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956273+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.7 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464294+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956430+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.6 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463254+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956401+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462980+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956358+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.4 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463896+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956390+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460531+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956267+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460834+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956284+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463126+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956364+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.1 4 0 4 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460637+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956297+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461160+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956319+0000 0 0 2026-02-22T23:29:27.222 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:34.518756+0000 30'100 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956260+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463001+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956353+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.e 3 0 3 0 0 2514 0 0 17 84 17 active+undersized+degraded 2026-02-22T23:28:19.464193+0000 47'101 49:144 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956424+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.464489+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956436+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.461149+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956314+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462131+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956335+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.12 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462505+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956342+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460903+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956330+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460764+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956290+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.460906+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956302+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.16 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462921+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956396+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463436+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956418+0000 0 0 2026-02-22T23:29:27.223 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.462643+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956348+0000 0 0 2026-02-22T23:29:27.224 INFO:tasks.workunit.client.0.trial122.stdout:1.19 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463337+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956374+0000 0 0 2026-02-22T23:29:27.224 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463532+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956385+0000 0 0 2026-02-22T23:29:27.224 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 3 0 3 0 0 2275 0 0 16 84 16 active+undersized+degraded 2026-02-22T23:28:19.463030+0000 30'100 49:143 [0,2] 0 [0,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-22T23:29:20.956369+0000 0 0 2026-02-22T23:29:27.224 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:27.224 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:27.224 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:803: TEST_divergent_3: activate_osd td/divergent-priors 1 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:29:27.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:29:27.232 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:27.233 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:27.233 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:29:27.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:29:27.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:29:27.235 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:29:27.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:29:27.258 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:27.253+0000 7fa0666ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:27.259 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:27.257+0000 7fa0666ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:27.260 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:27.257+0000 7fa0666ba8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:27.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:29:27.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:29:27.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:29:27.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:29:27.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:29:27.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:29:27.435 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:29:27.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:29:27.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:29:27.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:29:27.634 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 52 up_thru 52 down_at 32 last_clean_interval [9,30) [v2:127.0.0.1:6818/327770727,v1:127.0.0.1:6819/327770727] [v2:127.0.0.1:6820/327770727,v1:127.0.0.1:6821/327770727] exists,up cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:804: TEST_divergent_3: wait_for_osd up 1 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:29:27.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:29:27.636 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:29:27.636 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:29:27.636 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:29:27.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:29:27.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:29:27.637 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:29:27.833 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 52 up_thru 52 down_at 32 last_clean_interval [9,30) [v2:127.0.0.1:6818/327770727,v1:127.0.0.1:6819/327770727] [v2:127.0.0.1:6820/327770727,v1:127.0.0.1:6821/327770727] exists,up cc902fd6-3159-44fe-844d-063a2679a371 2026-02-22T23:29:27.833 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:29:27.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:29:27.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:29:27.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:806: TEST_divergent_3: sleep 20 2026-02-22T23:29:28.528 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:28.525+0000 7fa0666ba8c0 -1 Falling back to public interface 2026-02-22T23:29:31.503 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:31.501+0000 7fa0666ba8c0 -1 osd.1 53 log_to_monitors true 2026-02-22T23:29:47.834 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: get_asok_path osd.1 2026-02-22T23:29:47.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:29:47.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:29:47.835 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:29:47.835 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:47.835 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:47.836 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:29:47.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: CEPH_ARGS= 2026-02-22T23:29:47.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:807: TEST_divergent_3: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok dump_ops_in_flight 2026-02-22T23:29:47.896 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:29:47.896 INFO:tasks.workunit.client.0.trial122.stdout: "ops": [], 2026-02-22T23:29:47.896 INFO:tasks.workunit.client.0.trial122.stdout: "num_ops": 0 2026-02-22T23:29:47.896 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:29:47.902 INFO:tasks.workunit.client.0.trial122.stdout:allowing recovery 2026-02-22T23:29:47.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:809: TEST_divergent_3: echo 'allowing recovery' 2026-02-22T23:29:47.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:810: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:41.465023+0000 30'100 65:167 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:39.466405+0000 30'100 63:163 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467840+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:43.674759+0000 30'100 65:177 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:43.674696+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-24T00:39:07.414093+0000 3 0 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:39.466463+0000 30'100 63:163 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.9 5 0 0 0 0 3640 0 0 16 84 16 active+clean 2026-02-22T23:29:42.246731+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:42.246665+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:27:25.632353+0000 5 0 2026-02-22T23:29:48.100 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:34.579623+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:34.579554+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-23T23:31:13.122371+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.7 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:45.615591+0000 30'100 65:175 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:45.615517+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:15:22.639045+0000 2 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.6 4 0 0 0 0 3003 0 0 17 84 17 active+clean 2026-02-22T23:29:39.466466+0000 53'101 63:157 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467582+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.4 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:33.576932+0000 30'100 65:180 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:33.576862+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:48:02.237789+0000 2 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 0 0 0 2275 0 0 10 90 10 active+clean 2026-02-22T23:29:41.465019+0000 30'100 65:169 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:38.250534+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:39.466259+0000 30'100 63:163 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 0 0 0 2184 0 0 14 86 14 active+clean 2026-02-22T23:29:40.614412+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:40.614322+0000 30'100 2026-02-22T23:29:40.614322+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:46:33.137128+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.1 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:39.205485+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:39.205409+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:40:05.530571+0000 2 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:41.645154+0000 30'100 65:180 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:41.645064+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-24T03:05:50.236606+0000 2 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:37.568964+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:37.568897+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-23T23:46:19.001647+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467237+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.e 4 0 0 0 0 3151 0 0 17 84 17 active+clean 2026-02-22T23:29:40.206612+0000 47'101 63:172 [0,1,2] 0 [0,1,2] 0 47'101 2026-02-22T23:29:40.206549+0000 47'101 2026-02-22T23:29:40.206549+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:53:38.066272+0000 4 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:41.464682+0000 30'100 65:167 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:35.538639+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:35.538578+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:43:27.360360+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 0 0 0 2184 0 0 16 84 16 active+clean 2026-02-22T23:29:40.191429+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:40.191350+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:27:15.250572+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.12 1 0 0 0 0 728 0 0 16 84 16 active+clean 2026-02-22T23:29:43.275695+0000 30'100 63:171 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-22T23:29:43.275636+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:27:01.044824+0000 1 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:44.652345+0000 30'100 65:177 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:44.652291+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T07:17:13.774937+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:42.633410+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:42.633349+0000 30'100 2026-02-22T23:29:42.633349+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:25:58.894693+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467279+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:36.522787+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.16 4 0 0 0 0 2912 0 0 16 84 16 active+clean 2026-02-22T23:29:41.220146+0000 30'100 63:171 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-22T23:29:41.220064+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-24T03:44:20.899052+0000 4 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:41.464821+0000 30'100 65:167 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:39.631241+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:39.631159+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T07:28:08.863020+0000 3 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.19 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:41.196820+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:41.196734+0000 30'100 2026-02-22T23:29:41.196734+0000 0 0 periodic scrub scheduled @ 2026-02-24T11:03:44.841586+0000 2 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 4 0 0 0 0 2912 0 0 16 84 16 active+clean 2026-02-22T23:29:42.236105+0000 30'100 63:171 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-22T23:29:42.236037+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:08:59.372889+0000 4 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 5 0 0 0 0 3640 0 0 10 90 10 active+clean 2026-02-22T23:29:46.583742+0000 30'100 65:179 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:46.583679+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:49:33.628456+0000 5 0 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:48.101 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:48.109 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-22T23:29:48.109 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-22T23:29:48.192 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:29:48.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-22T23:29:48.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-22T23:29:48.276 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:29:48.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:812: TEST_divergent_3: for i in $osds 2026-02-22T23:29:48.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:814: TEST_divergent_3: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-22T23:29:48.361 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:29:48.368 INFO:tasks.workunit.client.0.trial122.stdout:reading divergent objects 2026-02-22T23:29:48.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:817: TEST_divergent_3: echo 'reading divergent objects' 2026-02-22T23:29:48.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:818: TEST_divergent_3: ceph pg dump pgs 2026-02-22T23:29:48.561 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:29:48.561 INFO:tasks.workunit.client.0.trial122.stdout:1.1f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:41.465023+0000 30'100 65:167 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.561 INFO:tasks.workunit.client.0.trial122.stdout:1.1e 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:39.466405+0000 30'100 63:163 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.1d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467840+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.b 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:43.674759+0000 30'100 65:177 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:43.674696+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-24T00:39:07.414093+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.a 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:39.466463+0000 30'100 63:163 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.9 5 0 0 0 0 3640 0 0 16 84 16 active+clean 2026-02-22T23:29:42.246731+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:42.246665+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T08:27:25.632353+0000 5 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.8 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:34.579623+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:34.579554+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-23T23:31:13.122371+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.7 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:45.615591+0000 30'100 65:175 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:45.615517+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:15:22.639045+0000 2 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.6 4 0 0 0 0 3003 0 0 17 84 17 active+clean 2026-02-22T23:29:39.466466+0000 53'101 63:157 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.5 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467582+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.4 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:33.576932+0000 30'100 65:180 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:33.576862+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:48:02.237789+0000 2 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.3 4 0 0 0 0 2275 0 0 10 90 10 active+clean 2026-02-22T23:29:41.465019+0000 30'100 65:169 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:38.250534+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.2 4 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:39.466259+0000 30'100 63:163 [0,1,2] 0 [0,1,2] 0 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.c 3 0 0 0 0 2184 0 0 14 86 14 active+clean 2026-02-22T23:29:40.614412+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:40.614322+0000 30'100 2026-02-22T23:29:40.614322+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:46:33.137128+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.1 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:39.205485+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:39.205409+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T03:40:05.530571+0000 2 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.1c 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:41.645154+0000 30'100 65:180 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:41.645064+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-24T03:05:50.236606+0000 2 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.0 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:37.568964+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:37.568897+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-23T23:46:19.001647+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.d 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467237+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.e 4 0 0 0 0 3151 0 0 17 84 17 active+clean 2026-02-22T23:29:40.206612+0000 47'101 63:172 [0,1,2] 0 [0,1,2] 0 47'101 2026-02-22T23:29:40.206549+0000 47'101 2026-02-22T23:29:40.206549+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:53:38.066272+0000 4 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.f 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:41.464682+0000 30'100 65:167 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.10 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:35.538639+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:35.538578+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:43:27.360360+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.11 3 0 0 0 0 2184 0 0 16 84 16 active+clean 2026-02-22T23:29:40.191429+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:40.191350+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:27:15.250572+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.12 1 0 0 0 0 728 0 0 16 84 16 active+clean 2026-02-22T23:29:43.275695+0000 30'100 63:171 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-22T23:29:43.275636+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T10:27:01.044824+0000 1 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.13 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:44.652345+0000 30'100 65:177 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:44.652291+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T07:17:13.774937+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.14 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:42.633410+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:42.633349+0000 30'100 2026-02-22T23:29:42.633349+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:25:58.894693+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.15 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:37.467279+0000 30'100 63:143 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:36.522787+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.16 4 0 0 0 0 2912 0 0 16 84 16 active+clean 2026-02-22T23:29:41.220146+0000 30'100 63:171 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-22T23:29:41.220064+0000 0'0 2026-02-22T23:27:42.773841+0000 0 1 periodic scrub scheduled @ 2026-02-24T03:44:20.899052+0000 4 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.17 3 0 0 0 0 2275 0 0 16 84 16 active+clean 2026-02-22T23:29:41.464821+0000 30'100 65:167 [1,2,0] 1 [1,2,0] 1 0'0 2026-02-22T23:27:42.773841+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 queued for scrub 0 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.18 3 0 0 0 0 2184 0 0 10 90 10 active+clean 2026-02-22T23:29:39.631241+0000 30'100 65:181 [1,0,2] 1 [1,0,2] 1 30'100 2026-02-22T23:29:39.631159+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T07:28:08.863020+0000 3 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.19 2 0 0 0 0 1456 0 0 16 84 16 active+clean 2026-02-22T23:29:41.196820+0000 30'100 63:151 [2,0,1] 2 [2,0,1] 2 30'100 2026-02-22T23:29:41.196734+0000 30'100 2026-02-22T23:29:41.196734+0000 0 0 periodic scrub scheduled @ 2026-02-24T11:03:44.841586+0000 2 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.1a 4 0 0 0 0 2912 0 0 16 84 16 active+clean 2026-02-22T23:29:42.236105+0000 30'100 63:171 [0,1,2] 0 [0,1,2] 0 30'100 2026-02-22T23:29:42.236037+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T01:08:59.372889+0000 4 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:1.1b 5 0 0 0 0 3640 0 0 10 90 10 active+clean 2026-02-22T23:29:46.583742+0000 30'100 65:179 [1,2,0] 1 [1,2,0] 1 30'100 2026-02-22T23:29:46.583679+0000 0'0 2026-02-22T23:27:42.773841+0000 0 0 periodic scrub scheduled @ 2026-02-24T06:49:33.628456+0000 5 0 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:29:48.562 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:29:48.571 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: expr 2 + 2 2026-02-22T23:29:48.572 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: seq 1 4 2026-02-22T23:29:48.573 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:29:48.573 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_1 td/divergent-priors/existing 2026-02-22T23:29:48.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:29:48.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_2 td/divergent-priors/existing 2026-02-22T23:29:48.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:29:48.622 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_3 td/divergent-priors/existing 2026-02-22T23:29:48.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:819: TEST_divergent_3: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:29:48.643 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:821: TEST_divergent_3: rados -p test get existing_4 td/divergent-priors/existing 2026-02-22T23:29:48.669 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: seq 1 2 2026-02-22T23:29:48.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:29:48.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get newobject_1 td/divergent-priors/existing 2026-02-22T23:29:48.688 INFO:tasks.workunit.client.0.trial122.stderr:error getting test/newobject_1: (2) No such file or directory 2026-02-22T23:29:48.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:823: TEST_divergent_3: for i in $(seq 1 $DIVERGENT_CREATE) 2026-02-22T23:29:48.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:825: TEST_divergent_3: rados -p test get newobject_2 td/divergent-priors/existing 2026-02-22T23:29:48.715 INFO:tasks.workunit.client.0.trial122.stderr:error getting test/newobject_2: (2) No such file or directory 2026-02-22T23:29:48.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:827: TEST_divergent_3: rm -f td/divergent-priors/existing 2026-02-22T23:29:48.719 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-22T23:29:48.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:829: TEST_divergent_3: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 20 _merge_object_divergent_entries: merging hoid 1:97db2a0f:::existing_2:head entries: 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4559.0:1 2026-02-22T23:27:55.608150+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 20 _merge_object_divergent_entries: keeping 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4559.0:1 2026-02-22T23:27:55.608150+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version: 30'2 first_divergent_update: 30'104 last_divergent_update: 30'104 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head has no more recent entries in log 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head olog_can_rollback_to: 30'106 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot rollback 30'104 (30'2) modify 1:97db2a0f:::existing_2:head by client.4559.0:1 2026-02-22T23:27:55.608150+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head cannot roll back, removing and adding to missing 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff147145640 10 _merge_object_divergent_entries: hoid 1:97db2a0f:::existing_2:head prior_version 30'2 <= info.log_tail 30'89 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 20 _merge_object_divergent_entries: merging hoid 1:70b3bf12:::existing_4:head entries: 30'106 (30'4) delete 1:70b3bf12:::existing_4:head by client.4565.0:1 2026-02-22T23:27:55.614040+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 20 _merge_object_divergent_entries: keeping 30'106 (30'4) delete 1:70b3bf12:::existing_4:head by client.4565.0:1 2026-02-22T23:27:55.614040+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: hoid 1:70b3bf12:::existing_4:head prior_version: 30'4 first_divergent_update: 30'106 last_divergent_update: 30'106 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: more recent entry found: 47'101 (30'4) modify 1:70b3bf12:::existing_4:head by client.4686.0:1 2026-02-22T23:28:47.662605+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false, already merged 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 20 _merge_object_divergent_entries: merging hoid 1:75ac5091:::newobject_2:head entries: 30'102 (0'0) modify 1:75ac5091:::newobject_2:head by client.4555.0:1 2026-02-22T23:27:55.607396+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 20 _merge_object_divergent_entries: keeping 30'102 (0'0) modify 1:75ac5091:::newobject_2:head by client.4555.0:1 2026-02-22T23:27:55.607396+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version: 0'0 first_divergent_update: 30'102 last_divergent_update: 30'102 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head has no more recent entries in log 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff148948640 10 _merge_object_divergent_entries: hoid 1:75ac5091:::newobject_2:head prior_version or op type indicates creation, deleting 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 20 _merge_object_divergent_entries: merging hoid 1:d858c035:::existing_1:head entries: 30'103 (30'1) modify 1:d858c035:::existing_1:head by client.4558.0:1 2026-02-22T23:27:55.608007+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 20 _merge_object_divergent_entries: keeping 30'103 (30'1) modify 1:d858c035:::existing_1:head by client.4558.0:1 2026-02-22T23:27:55.608007+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version: 30'1 first_divergent_update: 30'103 last_divergent_update: 30'103 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head has no more recent entries in log 2026-02-22T23:29:48.724 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:29:48.725 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head olog_can_rollback_to: 30'106 2026-02-22T23:29:48.725 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot rollback 30'103 (30'1) modify 1:d858c035:::existing_1:head by client.4558.0:1 2026-02-22T23:27:55.608007+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:29:48.725 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head cannot roll back, removing and adding to missing 2026-02-22T23:29:48.725 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:d858c035:::existing_1:head prior_version 30'1 <= info.log_tail 30'90 2026-02-22T23:29:48.725 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 20 _merge_object_divergent_entries: merging hoid 1:da59f6e7:::newobject_1:head entries: 30'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4557.0:1 2026-02-22T23:27:55.607500+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:29:48.725 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 20 _merge_object_divergent_entries: keeping 30'101 (0'0) modify 1:da59f6e7:::newobject_1:head by client.4557.0:1 2026-02-22T23:27:55.607500+0000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version: 0'0 first_divergent_update: 30'101 last_divergent_update: 30'101 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head has no more recent entries in log 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14a14b640 10 _merge_object_divergent_entries: hoid 1:da59f6e7:::newobject_1:head prior_version or op type indicates creation, deleting 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 20 _merge_object_divergent_entries: merging hoid 1:8881e6d8:::existing_3:head entries: 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4562.0:1 2026-02-22T23:27:55.611541+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 20 _merge_object_divergent_entries: keeping 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4562.0:1 2026-02-22T23:27:55.611541+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid object_not_in_store: 1 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version: 30'3 first_divergent_update: 30'105 last_divergent_update: 30'105 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head has no more recent entries in log 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head olog_can_rollback_to: 30'106 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot rollback 30'105 (30'3) delete 1:8881e6d8:::existing_3:head by client.4562.0:1 2026-02-22T23:27:55.611541+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: false, new_object: false 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head cannot roll back, removing and adding to missing 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:29:23.305+0000 7ff14b14d640 10 _merge_object_divergent_entries: hoid 1:8881e6d8:::existing_3:head prior_version 30'3 <= info.log_tail 30'84 2026-02-22T23:29:48.733 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-22T23:29:48.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:831: TEST_divergent_3: grep -q '_merge_object_divergent_entries: more recent entry found:' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:29:48.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:837: TEST_divergent_3: find td/divergent-priors -name '*osd*log' 2026-02-22T23:29:48.740 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:837: TEST_divergent_3: grep -q '_merge_object_divergent_entries.*prior_version or op type indicates creation' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:29:48.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:842: TEST_divergent_3: echo success 2026-02-22T23:29:48.743 INFO:tasks.workunit.client.0.trial122.stdout:success 2026-02-22T23:29:48.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:844: TEST_divergent_3: rm td/divergent-priors/td/divergent-priors/exp.20359.out 2026-02-22T23:29:48.744 INFO:tasks.workunit.client.0.trial122.stderr:rm: cannot remove 'td/divergent-priors/td/divergent-priors/exp.20359.out': No such file or directory 2026-02-22T23:29:48.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:846: TEST_divergent_3: delete_pool test 2026-02-22T23:29:48.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-22T23:29:48.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-22T23:29:48.985 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' does not exist 2026-02-22T23:29:48.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:847: TEST_divergent_3: kill_daemons td/divergent-priors 2026-02-22T23:29:48.995 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:29:48.995 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:29:48.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:29:48.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:29:48.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:29:57.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:29:57.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-22T23:29:57.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:29:57.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:29:57.314 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:29:57.314 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:29:57.315 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:29:57.315 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:29:57.315 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:29:57.315 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:29:57.321 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:29:57.321 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:29:57.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:29:57.323 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:29:57.324 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:29:57.324 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:29:57.324 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:29:57.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:29:57.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:29:57.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:29:57.326 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:29:57.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:29:57.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:29:57.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:29:57.356 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:29:57.356 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.356 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:29:57.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:29:57.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:29:57.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:44: run: for func in $funcs 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:45: run: setup td/divergent-priors 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/divergent-priors 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/divergent-priors 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:29:57.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:29:57.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:29:57.361 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:29:57.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:29:57.363 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:29:57.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:29:57.364 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:29:57.364 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:29:57.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:29:57.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:29:57.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:29:57.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:29:57.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:29:57.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:29:57.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:29:57.370 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:29:57.370 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.370 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.370 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:29:57.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:29:57.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:29:57.371 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/divergent-priors 2026-02-22T23:29:57.373 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:29:57.373 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.373 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.373 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.20359 2026-02-22T23:29:57.374 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:29:57.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-22T23:29:57.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/divergent-priors 1' TERM HUP INT 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:46: run: TEST_divergent_ec td/divergent-priors 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:228: TEST_divergent_ec: local dir=td/divergent-priors 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:231: TEST_divergent_ec: local dummyfile=/etc/fstab 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:232: TEST_divergent_ec: local dummyfile2=/etc/resolv.conf 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:234: TEST_divergent_ec: local num_osds=3 2026-02-22T23:29:57.376 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: expr 3 - 1 2026-02-22T23:29:57.377 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: seq 0 2 2026-02-22T23:29:57.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:235: TEST_divergent_ec: local 'osds=0 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:236: TEST_divergent_ec: run_mon td/divergent-priors a 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/divergent-priors 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/divergent-priors/a 2026-02-22T23:29:57.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/divergent-priors/a --run-dir=td/divergent-priors 2026-02-22T23:29:57.408 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:29:57.409 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:29:57.409 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:29:57.410 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:29:57.410 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.410 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.410 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:57.410 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/divergent-priors/a '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --mon-cluster-log-file=td/divergent-priors/log --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:29:57.442 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:29:57.443 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:29:57.443 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:29:57.444 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:29:57.444 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:29:57.446 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:29:57.446 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get fsid 2026-02-22T23:29:57.501 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:29:57.501 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:29:57.501 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:29:57.501 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:29:57.501 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:29:57.502 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-mon.a.asok 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:29:57.503 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.20359/ceph-mon.a.asok config get mon_host 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:237: TEST_divergent_ec: run_mgr td/divergent-priors x 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/divergent-priors 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/divergent-priors/x 2026-02-22T23:29:57.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:29:57.668 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:29:57.668 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:29:57.669 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:29:57.669 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:29:57.669 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.670 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:57.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:29:57.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/divergent-priors/x '--log-file=td/divergent-priors/$name.log' '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --run-dir=td/divergent-priors '--pid-file=td/divergent-priors/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:29:57.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 0 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:29:57.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:29:57.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:29:57.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:29:57.698 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:29:57.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:29:57.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 4c996602-e7fe-497f-b8c0-5a182a52c84b' 2026-02-22T23:29:57.699 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:29:57.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:29:57.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB1kZtpLKxLKhAAxwRpYuL9Ytc1mZfejQL/Ow== 2026-02-22T23:29:57.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB1kZtpLKxLKhAAxwRpYuL9Ytc1mZfejQL/Ow=="}' 2026-02-22T23:29:57.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 4c996602-e7fe-497f-b8c0-5a182a52c84b -i td/divergent-priors/0/new.json 2026-02-22T23:29:57.820 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:29:57.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/0/new.json 2026-02-22T23:29:57.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB1kZtpLKxLKhAAxwRpYuL9Ytc1mZfejQL/Ow== --osd-uuid 4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:29:57.847 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:57.844+0000 7f5287c7e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:57.848 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:57.844+0000 7f5287c7e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:57.849 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:57.844+0000 7f5287c7e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:29:57.850 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:57.844+0000 7f5287c7e8c0 -1 bdev(0x56552c0fdc00 td/divergent-priors/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:29:57.850 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:29:57.844+0000 7f5287c7e8c0 -1 bluestore(td/divergent-priors/0) _read_fsid unparsable uuid 2026-02-22T23:30:00.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/0/keyring 2026-02-22T23:30:00.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:30:00.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:30:00.681 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:30:00.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:30:00.959 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:30:00.960 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:30:00.961 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:30:00.961 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:30:00.961 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:30:00.961 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:30:00.977 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:00.972+0000 7fa4305f78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:00.978 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:00.976+0000 7fa4305f78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:00.979 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:00.976+0000 7fa4305f78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:01.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:30:01.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:30:01.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:30:01.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:30:01.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:30:01.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:01.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:30:01.155 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:30:01.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:01.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:01.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:02.362 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:30:02.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:30:02.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:02.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:02.476 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:02.472+0000 7fa4305f78c0 -1 Falling back to public interface 2026-02-22T23:30:02.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:03.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:03.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:03.569 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:30:03.569 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:30:03.569 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:03.570 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:03.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:04.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:04.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:04.765 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:30:04.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:30:04.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:04.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:04.966 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:05.465 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:05.460+0000 7fa4305f78c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:30:05.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:05.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:05.967 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:30:05.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:30:05.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:05.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:06.168 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/809837995,v1:127.0.0.1:6803/809837995] [v2:127.0.0.1:6804/809837995,v1:127.0.0.1:6805/809837995] exists,up 4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:30:06.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:30:06.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:30:06.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:30:06.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 1 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:30:06.173 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:30:06.174 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:30:06.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:30:06.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:30:06.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:30:06.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:30:06.176 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:30:06.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=b2449376-0236-402d-9cb6-140a74a80dee 2026-02-22T23:30:06.176 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 b2449376-0236-402d-9cb6-140a74a80dee 2026-02-22T23:30:06.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 b2449376-0236-402d-9cb6-140a74a80dee' 2026-02-22T23:30:06.177 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:30:06.186 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQB+kZtpUdcTCxAAIwddHhRw3DmvQGKt+iXG4w== 2026-02-22T23:30:06.186 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQB+kZtpUdcTCxAAIwddHhRw3DmvQGKt+iXG4w=="}' 2026-02-22T23:30:06.186 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new b2449376-0236-402d-9cb6-140a74a80dee -i td/divergent-priors/1/new.json 2026-02-22T23:30:06.385 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:30:06.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/1/new.json 2026-02-22T23:30:06.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB+kZtpUdcTCxAAIwddHhRw3DmvQGKt+iXG4w== --osd-uuid b2449376-0236-402d-9cb6-140a74a80dee 2026-02-22T23:30:06.413 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:06.408+0000 7fcdf470a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:06.415 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:06.412+0000 7fcdf470a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:06.416 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:06.412+0000 7fcdf470a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:06.416 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:06.412+0000 7fcdf470a8c0 -1 bdev(0x56535d429c00 td/divergent-priors/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:30:06.417 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:06.412+0000 7fcdf470a8c0 -1 bluestore(td/divergent-priors/1) _read_fsid unparsable uuid 2026-02-22T23:30:08.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/1/keyring 2026-02-22T23:30:08.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:30:08.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:30:08.680 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:30:08.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:30:08.970 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:30:08.970 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:30:08.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:30:08.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:30:08.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:30:08.971 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:30:08.997 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:08.992+0000 7ffb83ea78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:08.999 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:08.996+0000 7ffb83ea78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:09.000 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:08.996+0000 7ffb83ea78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:09.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:30:09.172 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:30:09.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:09.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:30:09.374 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:10.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:10.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:10.376 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:30:10.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:30:10.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:10.377 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:30:10.492 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:10.488+0000 7ffb83ea78c0 -1 Falling back to public interface 2026-02-22T23:30:10.575 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:11.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:11.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:11.577 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:30:11.577 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:30:11.578 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:11.578 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:30:11.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:12.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:12.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:12.776 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:30:12.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:30:12.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:12.777 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:30:12.979 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:13.486 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:13.480+0000 7ffb83ea78c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:30:13.980 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:13.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:13.981 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:30:13.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:30:13.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:13.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:30:14.176 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2569739617,v1:127.0.0.1:6811/2569739617] [v2:127.0.0.1:6812/2569739617,v1:127.0.0.1:6813/2569739617] exists,up b2449376-0236-402d-9cb6-140a74a80dee 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:238: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:240: TEST_divergent_ec: run_osd td/divergent-priors 2 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/divergent-priors 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:30:14.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:30:14.181 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:30:14.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:30:14.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:30:14.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:30:14.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:30:14.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:30:14.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:30:14.183 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:30:14.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:30:14.184 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:30:14.184 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32' 2026-02-22T23:30:14.184 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:30:14.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCGkZtpwyWmCxAAk3xgFP1yrHlyHKPUm0jXow== 2026-02-22T23:30:14.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCGkZtpwyWmCxAAk3xgFP1yrHlyHKPUm0jXow=="}' 2026-02-22T23:30:14.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 -i td/divergent-priors/2/new.json 2026-02-22T23:30:14.398 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:30:14.406 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/divergent-priors/2/new.json 2026-02-22T23:30:14.407 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCGkZtpwyWmCxAAk3xgFP1yrHlyHKPUm0jXow== --osd-uuid fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:30:14.426 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:14.420+0000 7fc297b188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:14.427 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:14.424+0000 7fc297b188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:14.428 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:14.424+0000 7fc297b188c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:14.429 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:14.424+0000 7fc297b188c0 -1 bdev(0x55602ace3c00 td/divergent-priors/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:30:14.429 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:14.424+0000 7fc297b188c0 -1 bluestore(td/divergent-priors/2) _read_fsid unparsable uuid 2026-02-22T23:30:16.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/divergent-priors/2/keyring 2026-02-22T23:30:16.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:30:16.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:30:16.693 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:30:16.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/divergent-priors/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:30:16.972 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:30:16.973 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:30:16.973 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:30:16.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:30:16.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:30:16.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:30:17.000 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:16.996+0000 7f3aa9d058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:17.001 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:16.996+0000 7f3aa9d058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:17.002 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:16.996+0000 7f3aa9d058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:17.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:30:17.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:30:17.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:30:17.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:30:17.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:30:17.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:17.177 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:30:17.178 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:30:17.178 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:17.178 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:17.376 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:18.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:18.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:18.378 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:30:18.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:30:18.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:18.379 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:18.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:18.796 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:18.792+0000 7f3aa9d058c0 -1 Falling back to public interface 2026-02-22T23:30:19.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:19.587 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:19.587 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:30:19.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:30:19.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:19.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:19.793 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:20.794 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:20.794 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:20.795 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:30:20.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:30:20.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:20.796 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:20.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:21.835 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:21.832+0000 7f3aa9d058c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:30:21.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:21.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:21.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:30:21.995 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:30:21.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:21.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:22.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:23.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:23.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:23.193 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:30:23.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:30:23.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:23.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:23.392 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1680730599,v1:127.0.0.1:6819/1680730599] [v2:127.0.0.1:6820/1680730599,v1:127.0.0.1:6821/1680730599] exists,up fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:30:23.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:30:23.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:30:23.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:30:23.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:243: TEST_divergent_ec: ceph osd set noout 2026-02-22T23:30:23.690 INFO:tasks.workunit.client.0.trial122.stderr:noout is set 2026-02-22T23:30:23.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:244: TEST_divergent_ec: ceph osd set noin 2026-02-22T23:30:23.994 INFO:tasks.workunit.client.0.trial122.stderr:noin is set 2026-02-22T23:30:24.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:245: TEST_divergent_ec: ceph osd set nodown 2026-02-22T23:30:24.298 INFO:tasks.workunit.client.0.trial122.stderr:nodown is set 2026-02-22T23:30:24.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:246: TEST_divergent_ec: create_ec_pool test true k=2 m=1 2026-02-22T23:30:24.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2498: create_ec_pool: local pool_name=test 2026-02-22T23:30:24.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: create_ec_pool: shift 2026-02-22T23:30:24.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: create_ec_pool: local allow_overwrites=true 2026-02-22T23:30:24.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: create_ec_pool: shift 2026-02-22T23:30:24.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: create_ec_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd k=2 m=1 2026-02-22T23:30:24.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2505: create_ec_pool: create_pool test 1 1 erasure myprofile 2026-02-22T23:30:24.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create test 1 1 erasure myprofile 2026-02-22T23:30:24.953 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' already exists 2026-02-22T23:30:24.962 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:30:25.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2507: create_ec_pool: '[' true = true ']' 2026-02-22T23:30:25.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2508: create_ec_pool: ceph osd pool set test allow_ec_overwrites true 2026-02-22T23:30:26.295 INFO:tasks.workunit.client.0.trial122.stderr:set pool 1 allow_ec_overwrites to true 2026-02-22T23:30:26.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2511: create_ec_pool: wait_for_clean 2026-02-22T23:30:26.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:30:26.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:30:26.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:30:26.310 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:30:26.310 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:30:26.311 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:30:26.311 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:30:26.311 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:30:26.311 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:30:26.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:30:26.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:30:26.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:30:26.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:30:26.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:30:26.414 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:30:26.608 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:30:26.608 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:30:26.609 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:30:26.609 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:30:26.609 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:26.609 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:30:26.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836487 2026-02-22T23:30:26.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836487 2026-02-22T23:30:26.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487' 2026-02-22T23:30:26.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:26.689 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:30:26.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705669 2026-02-22T23:30:26.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705669 2026-02-22T23:30:26.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669' 2026-02-22T23:30:26.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:26.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:30:26.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542147 2026-02-22T23:30:26.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542147 2026-02-22T23:30:26.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836487 1-38654705669 2-60129542147' 2026-02-22T23:30:26.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:26.846 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836487 2026-02-22T23:30:26.846 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:26.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:30:26.847 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836487 2026-02-22T23:30:26.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:26.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836487 2026-02-22T23:30:26.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836487' 2026-02-22T23:30:26.849 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836487 2026-02-22T23:30:26.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:27.050 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836484 -lt 21474836487 2026-02-22T23:30:27.050 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:30:28.051 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:30:28.052 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:28.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836487 -lt 21474836487 2026-02-22T23:30:28.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:28.254 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705669 2026-02-22T23:30:28.254 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:28.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:30:28.256 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705669 2026-02-22T23:30:28.256 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:28.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705669 2026-02-22T23:30:28.258 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705669 2026-02-22T23:30:28.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705669' 2026-02-22T23:30:28.258 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:30:28.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705669 -lt 38654705669 2026-02-22T23:30:28.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:28.468 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542147 2026-02-22T23:30:28.468 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:28.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:30:28.470 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542147 2026-02-22T23:30:28.470 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:28.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542147 2026-02-22T23:30:28.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542147' 2026-02-22T23:30:28.471 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542147 2026-02-22T23:30:28.472 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:30:28.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542147 -lt 60129542147 2026-02-22T23:30:28.685 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:30:28.686 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:30:28.686 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:30:28.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:30:28.947 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:30:29.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:30:29.141 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:30:29.142 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:30:29.142 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:30:29.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:30:29.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:30:29.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:30:29.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2512: create_ec_pool: return 0 2026-02-22T23:30:29.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:248: TEST_divergent_ec: flush_pg_stats 2026-02-22T23:30:29.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:30:29.400 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:30:29.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:30:29.604 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:30:29.604 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:30:29.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:30:29.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:29.604 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:30:29.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836489 2026-02-22T23:30:29.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836489 2026-02-22T23:30:29.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489' 2026-02-22T23:30:29.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:29.683 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:30:29.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705672 2026-02-22T23:30:29.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705672 2026-02-22T23:30:29.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672' 2026-02-22T23:30:29.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:29.764 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:30:29.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542150 2026-02-22T23:30:29.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542150 2026-02-22T23:30:29.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836489 1-38654705672 2-60129542150' 2026-02-22T23:30:29.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:29.844 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836489 2026-02-22T23:30:29.844 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:29.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:30:29.846 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836489 2026-02-22T23:30:29.846 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:29.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836489 2026-02-22T23:30:29.847 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836489 2026-02-22T23:30:29.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836489' 2026-02-22T23:30:29.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:30.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836489 2026-02-22T23:30:30.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:30.043 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705672 2026-02-22T23:30:30.044 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:30.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:30:30.045 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705672 2026-02-22T23:30:30.046 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:30.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705672 2026-02-22T23:30:30.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-22T23:30:30.047 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705672 2026-02-22T23:30:30.048 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:30:30.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-22T23:30:30.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:30.250 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542150 2026-02-22T23:30:30.250 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:30.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:30:30.252 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542150 2026-02-22T23:30:30.252 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:30.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542150 2026-02-22T23:30:30.254 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542150' 2026-02-22T23:30:30.254 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542150 2026-02-22T23:30:30.254 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:30:30.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542150 -lt 60129542150 2026-02-22T23:30:30.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:249: TEST_divergent_ec: wait_for_clean 2026-02-22T23:30:30.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:30:30.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:30:30.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:30:30.461 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:30:30.462 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:30:30.462 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:30:30.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:30:30.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:30:30.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:30:30.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:30:30.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:30:30.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:30:30.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:30:30.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:30:30.568 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:30:30.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:30:30.765 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:30:30.766 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:30:30.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:30:30.766 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:30.766 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:30:30.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836491 2026-02-22T23:30:30.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836491 2026-02-22T23:30:30.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491' 2026-02-22T23:30:30.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:30.845 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:30:30.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705674 2026-02-22T23:30:30.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705674 2026-02-22T23:30:30.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-38654705674' 2026-02-22T23:30:30.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:30.925 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:30:31.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542152 2026-02-22T23:30:31.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542152 2026-02-22T23:30:31.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836491 1-38654705674 2-60129542152' 2026-02-22T23:30:31.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:31.005 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836491 2026-02-22T23:30:31.005 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:31.006 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:30:31.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836491 2026-02-22T23:30:31.007 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:31.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836491 2026-02-22T23:30:31.009 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836491 2026-02-22T23:30:31.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836491' 2026-02-22T23:30:31.009 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:31.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 21474836491 2026-02-22T23:30:31.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:30:32.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:30:32.207 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:32.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836491 2026-02-22T23:30:32.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:32.409 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705674 2026-02-22T23:30:32.409 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:32.410 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:30:32.411 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705674 2026-02-22T23:30:32.411 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:32.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705674 2026-02-22T23:30:32.413 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705674 2026-02-22T23:30:32.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705674' 2026-02-22T23:30:32.413 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:30:32.623 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705674 -lt 38654705674 2026-02-22T23:30:32.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:32.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542152 2026-02-22T23:30:32.624 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:32.625 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:30:32.626 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542152 2026-02-22T23:30:32.626 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:32.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542152 2026-02-22T23:30:32.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542152' 2026-02-22T23:30:32.628 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542152 2026-02-22T23:30:32.628 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:30:32.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542152 -lt 60129542152 2026-02-22T23:30:32.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:30:32.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:30:32.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:30:33.092 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:30:33.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:30:33.287 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:30:33.288 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:30:33.288 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:30:33.547 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:30:33.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:30:33.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:30:33.549 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: ceph pg dump pgs --format=json 2026-02-22T23:30:33.549 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: jq '.pg_stats[0].up_primary' 2026-02-22T23:30:33.733 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:30:33.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:252: TEST_divergent_ec: local divergent=1 2026-02-22T23:30:33.743 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:253: TEST_divergent_ec: echo 'primary and soon to be divergent is 1' 2026-02-22T23:30:33.743 INFO:tasks.workunit.client.0.trial122.stdout:primary and soon to be divergent is 1 2026-02-22T23:30:33.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:254: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:30:33.932 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:30:33.932 INFO:tasks.workunit.client.0.trial122.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-22T23:30:25.790872+0000 0'0 27:17 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:22:01.627210+0000 0 0 2026-02-22T23:30:33.932 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:30:33.932 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:30:33.932 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:30:33.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:255: TEST_divergent_ec: local non_divergent= 2026-02-22T23:30:33.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:33.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 0 = 1 ']' 2026-02-22T23:30:33.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0' 2026-02-22T23:30:33.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:33.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 1 = 1 ']' 2026-02-22T23:30:33.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:259: TEST_divergent_ec: continue 2026-02-22T23:30:33.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:256: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:33.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:258: TEST_divergent_ec: '[' 2 = 1 ']' 2026-02-22T23:30:33.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:261: TEST_divergent_ec: non_divergent=' 0 2' 2026-02-22T23:30:33.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:264: TEST_divergent_ec: echo 'writing initial objects' 2026-02-22T23:30:33.943 INFO:tasks.workunit.client.0.trial122.stdout:writing initial objects 2026-02-22T23:30:33.943 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: seq 1 100 2026-02-22T23:30:33.943 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:33.943 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_1 /etc/fstab 2026-02-22T23:30:33.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:33.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_2 /etc/fstab 2026-02-22T23:30:33.983 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:33.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_3 /etc/fstab 2026-02-22T23:30:34.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_4 /etc/fstab 2026-02-22T23:30:34.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_5 /etc/fstab 2026-02-22T23:30:34.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_6 /etc/fstab 2026-02-22T23:30:34.096 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.096 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_7 /etc/fstab 2026-02-22T23:30:34.123 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.123 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_8 /etc/fstab 2026-02-22T23:30:34.151 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.151 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_9 /etc/fstab 2026-02-22T23:30:34.171 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_10 /etc/fstab 2026-02-22T23:30:34.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_11 /etc/fstab 2026-02-22T23:30:34.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.219 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_12 /etc/fstab 2026-02-22T23:30:34.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_13 /etc/fstab 2026-02-22T23:30:34.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_14 /etc/fstab 2026-02-22T23:30:34.303 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_15 /etc/fstab 2026-02-22T23:30:34.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_16 /etc/fstab 2026-02-22T23:30:34.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_17 /etc/fstab 2026-02-22T23:30:34.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_18 /etc/fstab 2026-02-22T23:30:34.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_19 /etc/fstab 2026-02-22T23:30:34.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_20 /etc/fstab 2026-02-22T23:30:34.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_21 /etc/fstab 2026-02-22T23:30:34.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_22 /etc/fstab 2026-02-22T23:30:34.520 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.520 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_23 /etc/fstab 2026-02-22T23:30:34.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_24 /etc/fstab 2026-02-22T23:30:34.570 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.570 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_25 /etc/fstab 2026-02-22T23:30:34.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_26 /etc/fstab 2026-02-22T23:30:34.611 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_27 /etc/fstab 2026-02-22T23:30:34.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_28 /etc/fstab 2026-02-22T23:30:34.665 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.665 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_29 /etc/fstab 2026-02-22T23:30:34.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_30 /etc/fstab 2026-02-22T23:30:34.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_31 /etc/fstab 2026-02-22T23:30:34.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_32 /etc/fstab 2026-02-22T23:30:34.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_33 /etc/fstab 2026-02-22T23:30:34.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_34 /etc/fstab 2026-02-22T23:30:34.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_35 /etc/fstab 2026-02-22T23:30:34.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_36 /etc/fstab 2026-02-22T23:30:34.866 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.866 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_37 /etc/fstab 2026-02-22T23:30:34.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.896 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_38 /etc/fstab 2026-02-22T23:30:34.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_39 /etc/fstab 2026-02-22T23:30:34.950 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.950 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_40 /etc/fstab 2026-02-22T23:30:34.978 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.979 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_41 /etc/fstab 2026-02-22T23:30:34.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:34.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_42 /etc/fstab 2026-02-22T23:30:35.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.026 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_43 /etc/fstab 2026-02-22T23:30:35.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_44 /etc/fstab 2026-02-22T23:30:35.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_45 /etc/fstab 2026-02-22T23:30:35.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_46 /etc/fstab 2026-02-22T23:30:35.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_47 /etc/fstab 2026-02-22T23:30:35.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_48 /etc/fstab 2026-02-22T23:30:35.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_49 /etc/fstab 2026-02-22T23:30:35.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_50 /etc/fstab 2026-02-22T23:30:35.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_51 /etc/fstab 2026-02-22T23:30:35.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_52 /etc/fstab 2026-02-22T23:30:35.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_53 /etc/fstab 2026-02-22T23:30:35.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_54 /etc/fstab 2026-02-22T23:30:35.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_55 /etc/fstab 2026-02-22T23:30:35.380 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.381 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_56 /etc/fstab 2026-02-22T23:30:35.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_57 /etc/fstab 2026-02-22T23:30:35.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_58 /etc/fstab 2026-02-22T23:30:35.456 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.456 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_59 /etc/fstab 2026-02-22T23:30:35.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_60 /etc/fstab 2026-02-22T23:30:35.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_61 /etc/fstab 2026-02-22T23:30:35.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_62 /etc/fstab 2026-02-22T23:30:35.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_63 /etc/fstab 2026-02-22T23:30:35.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_64 /etc/fstab 2026-02-22T23:30:35.614 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.614 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_65 /etc/fstab 2026-02-22T23:30:35.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_66 /etc/fstab 2026-02-22T23:30:35.661 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.661 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_67 /etc/fstab 2026-02-22T23:30:35.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_68 /etc/fstab 2026-02-22T23:30:35.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_69 /etc/fstab 2026-02-22T23:30:35.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_70 /etc/fstab 2026-02-22T23:30:35.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_71 /etc/fstab 2026-02-22T23:30:35.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_72 /etc/fstab 2026-02-22T23:30:35.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_73 /etc/fstab 2026-02-22T23:30:35.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_74 /etc/fstab 2026-02-22T23:30:35.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_75 /etc/fstab 2026-02-22T23:30:35.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_76 /etc/fstab 2026-02-22T23:30:35.931 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.931 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_77 /etc/fstab 2026-02-22T23:30:35.959 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.959 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_78 /etc/fstab 2026-02-22T23:30:35.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:35.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_79 /etc/fstab 2026-02-22T23:30:36.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_80 /etc/fstab 2026-02-22T23:30:36.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_81 /etc/fstab 2026-02-22T23:30:36.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_82 /etc/fstab 2026-02-22T23:30:36.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.094 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_83 /etc/fstab 2026-02-22T23:30:36.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_84 /etc/fstab 2026-02-22T23:30:36.148 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.148 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_85 /etc/fstab 2026-02-22T23:30:36.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_86 /etc/fstab 2026-02-22T23:30:36.197 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.197 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_87 /etc/fstab 2026-02-22T23:30:36.224 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.224 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_88 /etc/fstab 2026-02-22T23:30:36.251 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_89 /etc/fstab 2026-02-22T23:30:36.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_90 /etc/fstab 2026-02-22T23:30:36.307 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.307 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_91 /etc/fstab 2026-02-22T23:30:36.335 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.335 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_92 /etc/fstab 2026-02-22T23:30:36.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_93 /etc/fstab 2026-02-22T23:30:36.391 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.391 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_94 /etc/fstab 2026-02-22T23:30:36.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_95 /etc/fstab 2026-02-22T23:30:36.447 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.447 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_96 /etc/fstab 2026-02-22T23:30:36.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_97 /etc/fstab 2026-02-22T23:30:36.503 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.503 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_98 /etc/fstab 2026-02-22T23:30:36.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2026-02-22T23:30:36.552 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:266: TEST_divergent_ec: for i in $(seq 1 $testobjects) 2026-02-22T23:30:36.552 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:268: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2026-02-22T23:30:36.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:30:36.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:271: TEST_divergent_ec: wait_for_clean 2026-02-22T23:30:36.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:30:36.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:30:36.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:30:36.580 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:30:36.581 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:30:36.581 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:30:36.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:30:36.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:30:36.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:30:36.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:30:36.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:30:36.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:30:36.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:30:36.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:30:36.655 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:30:36.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:30:36.857 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:30:36.857 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:30:36.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:30:36.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:36.858 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:30:36.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=21474836495 2026-02-22T23:30:36.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 21474836495 2026-02-22T23:30:36.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495' 2026-02-22T23:30:36.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:36.937 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:30:37.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=38654705677 2026-02-22T23:30:37.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 38654705677 2026-02-22T23:30:37.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-38654705677' 2026-02-22T23:30:37.016 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:30:37.016 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:30:37.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=60129542155 2026-02-22T23:30:37.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 60129542155 2026-02-22T23:30:37.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-21474836495 1-38654705677 2-60129542155' 2026-02-22T23:30:37.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:37.095 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-21474836495 2026-02-22T23:30:37.095 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:37.097 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:30:37.097 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-21474836495 2026-02-22T23:30:37.098 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:37.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=21474836495 2026-02-22T23:30:37.099 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 21474836495 2026-02-22T23:30:37.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-22T23:30:37.100 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:37.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836492 -lt 21474836495 2026-02-22T23:30:37.304 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:30:38.305 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:30:38.306 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:30:38.504 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836495 -lt 21474836495 2026-02-22T23:30:38.505 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:38.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-38654705677 2026-02-22T23:30:38.505 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:38.506 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:30:38.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-38654705677 2026-02-22T23:30:38.507 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:38.508 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=38654705677 2026-02-22T23:30:38.509 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 38654705677 2026-02-22T23:30:38.509 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 38654705677' 2026-02-22T23:30:38.509 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:30:38.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 38654705677 -lt 38654705677 2026-02-22T23:30:38.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:30:38.721 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-60129542155 2026-02-22T23:30:38.721 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:30:38.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:30:38.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-60129542155 2026-02-22T23:30:38.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:30:38.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=60129542155 2026-02-22T23:30:38.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 60129542155' 2026-02-22T23:30:38.725 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 60129542155 2026-02-22T23:30:38.725 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:30:38.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 60129542155 -lt 60129542155 2026-02-22T23:30:38.924 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:30:38.925 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:30:38.925 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:30:39.190 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:30:39.191 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:30:39.381 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:30:39.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:30:39.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:30:39.382 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:30:39.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:30:39.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:30:39.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:30:39.635 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: get_pg test existing_1 2026-02-22T23:30:39.635 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1093: get_pg: local poolname=test 2026-02-22T23:30:39.636 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1094: get_pg: local objectname=existing_1 2026-02-22T23:30:39.636 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: ceph --format json osd map test existing_1 2026-02-22T23:30:39.636 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1096: get_pg: jq -r .pgid 2026-02-22T23:30:39.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:273: TEST_divergent_ec: local pgid=1.0 2026-02-22T23:30:39.837 INFO:tasks.workunit.client.0.trial122.stdout:blackholing osds 0 2 2026-02-22T23:30:39.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:276: TEST_divergent_ec: echo 'blackholing osds 0 2' 2026-02-22T23:30:39.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:277: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:30:40.030 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:30:40.030 INFO:tasks.workunit.client.0.trial122.stdout:1.0 100 0 0 0 0 72800 0 0 16 84 16 active+clean 2026-02-22T23:30:25.790872+0000 27'100 27:117 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T09:22:01.627210+0000 0 0 2026-02-22T23:30:40.030 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:30:40.030 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:30:40.031 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:30:40.039 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:30:40.039 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.0 2026-02-22T23:30:40.040 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:30:40.040 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:30:40.041 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:30:40.041 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:30:40.041 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:30:40.041 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:30:40.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2026-02-22T23:30:40.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok config set objectstore_blackhole 1 2026-02-22T23:30:40.092 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:30:40.093 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:30:40.093 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:30:40.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:278: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:30:40.099 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: get_asok_path osd.2 2026-02-22T23:30:40.099 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:30:40.099 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:30:40.100 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:30:40.100 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:30:40.100 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:30:40.100 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:30:40.100 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: CEPH_ARGS= 2026-02-22T23:30:40.100 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:280: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok config set objectstore_blackhole 1 2026-02-22T23:30:40.160 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:30:40.160 INFO:tasks.workunit.client.0.trial122.stdout: "success": "objectstore_blackhole = '' (not observed, change may require restart) " 2026-02-22T23:30:40.161 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:30:40.166 INFO:tasks.workunit.client.0.trial122.stdout:writing divergent object 2026-02-22T23:30:40.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:284: TEST_divergent_ec: echo 'writing divergent object' 2026-02-22T23:30:40.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:286: TEST_divergent_ec: sleep 1 2026-02-22T23:30:40.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:285: TEST_divergent_ec: rados -p test put existing_100 /etc/resolv.conf 2026-02-22T23:30:41.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:288: TEST_divergent_ec: rados -p test mksnap snap1 2026-02-22T23:30:41.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:287: TEST_divergent_ec: rados -p test put existing_100 /etc/fstab 2026-02-22T23:30:41.240 INFO:tasks.workunit.client.0.trial122.stdout:created pool test snap snap1 2026-02-22T23:30:41.242 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:290: TEST_divergent_ec: sleep 10 2026-02-22T23:30:41.243 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: expr 100 - 1 2026-02-22T23:30:41.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:289: TEST_divergent_ec: rados -p test put existing_99 /etc/fstab 2026-02-22T23:30:51.244 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:291: TEST_divergent_ec: killall -9 rados 2026-02-22T23:30:51.293 INFO:tasks.workunit.client.0.trial122.stderr:rados: no process found 2026-02-22T23:30:51.293 INFO:tasks.workunit.client.0.trial122.stdout:killing all the osds 2026-02-22T23:30:51.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:294: TEST_divergent_ec: echo 'killing all the osds' 2026-02-22T23:30:51.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:295: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:30:51.500 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:30:51.500 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73528 0 0 14 90 14 active+clean 2026-02-22T23:30:25.790872+0000 28'104 28:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:54:33.654791+0000 0 0 2026-02-22T23:30:51.500 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:30:51.500 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:30:51.501 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:30:51.508 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:296: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd 2026-02-22T23:30:51.509 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:30:51.509 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:30:51.509 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:30:51.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:30:51.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:30:51.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:30:51.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:51.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.0 2026-02-22T23:30:51.902 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:30:51.911 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:51.912 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.1 2026-02-22T23:30:52.183 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:30:52.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:297: TEST_divergent_ec: for i in $osds 2026-02-22T23:30:52.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:299: TEST_divergent_ec: ceph osd down osd.2 2026-02-22T23:30:52.461 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:30:52.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:30:52.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.0 2026-02-22T23:30:52.734 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already out. 2026-02-22T23:30:52.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:301: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:30:52.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:303: TEST_divergent_ec: ceph osd out osd.2 2026-02-22T23:30:52.990 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already out. 2026-02-22T23:30:53.001 INFO:tasks.workunit.client.0.trial122.stdout:bringing up non_divergent 0 2 2026-02-22T23:30:53.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:307: TEST_divergent_ec: echo 'bringing up non_divergent 0 2' 2026-02-22T23:30:53.002 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:308: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:30:53.195 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:30:53.195 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73528 0 0 14 90 14 stale+active+clean 2026-02-22T23:30:25.790872+0000 28'104 28:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:54:33.654791+0000 0 0 2026-02-22T23:30:53.195 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:30:53.195 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:30:53.196 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:30:53.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:30:53.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-22T23:30:53.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:30:53.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:30:53.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:30:53.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:30:53.210 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:30:53.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:30:53.233 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:53.231+0000 7fe42ee938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:53.234 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:53.231+0000 7fe42ee938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:53.235 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:53.235+0000 7fe42ee938c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:53.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:30:53.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:30:53.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:30:53.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:30:53.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:30:53.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:53.410 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:30:53.410 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:30:53.410 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:53.410 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:53.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:54.480 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:54.479+0000 7fe42ee938c0 -1 Falling back to public interface 2026-02-22T23:30:54.614 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:54.614 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:54.615 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:30:54.615 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:30:54.616 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:54.616 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:54.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:55.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:55.820 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:55.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:30:55.821 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:30:55.821 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:55.822 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:56.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:57.030 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:57.030 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:57.030 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:30:57.030 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:30:57.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:57.031 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:57.234 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:57.531 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:57.531+0000 7fe42ee938c0 -1 osd.0 27 log_to_monitors true 2026-02-22T23:30:58.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:58.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:58.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:30:58.236 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:30:58.237 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:58.237 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:30:58.440 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up out weight 0 up_from 35 up_thru 0 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6802/1405372986,v1:127.0.0.1:6803/1405372986] [v2:127.0.0.1:6804/1405372986,v1:127.0.0.1:6805/1405372986] exists,up 4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:30:58.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:30:58.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:30:58.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:30:58.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:309: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:30:58.443 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:311: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:30:58.444 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:30:58.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:30:58.446 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:30:58.447 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:30:58.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:30:58.448 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-22T23:30:58.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:30:58.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:30:58.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:30:58.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:30:58.470 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:58.467+0000 7f71e9fbb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:58.471 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:58.471+0000 7f71e9fbb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:58.472 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:58.471+0000 7f71e9fbb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:58.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:30:58.641 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:30:58.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:58.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:30:58.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:30:59.740 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:30:59.739+0000 7f71e9fbb8c0 -1 Falling back to public interface 2026-02-22T23:30:59.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:30:59.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:30:59.846 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:30:59.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:30:59.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:30:59.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:31:00.051 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:31:01.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:31:01.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:31:01.053 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:31:01.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:31:01.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:31:01.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:31:01.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:31:02.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:31:02.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:31:02.255 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:31:02.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:31:02.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:31:02.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:31:02.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:31:02.833 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:02.831+0000 7f71e9fbb8c0 -1 osd.2 27 log_to_monitors true 2026-02-22T23:31:03.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:31:03.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:31:03.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:31:03.454 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:31:03.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:31:03.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:31:03.655 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up out weight 0 up_from 38 up_thru 0 down_at 31 last_clean_interval [14,27) [v2:127.0.0.1:6810/43963801,v1:127.0.0.1:6811/43963801] [v2:127.0.0.1:6812/43963801,v1:127.0.0.1:6813/43963801] exists,up fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:31:03.656 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:31:03.656 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:31:03.656 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:31:03.656 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:31:03.656 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.0 2026-02-22T23:31:03.909 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already in. 2026-02-22T23:31:03.919 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:313: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:31:03.919 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:315: TEST_divergent_ec: ceph osd in osd.2 2026-02-22T23:31:04.179 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already in. 2026-02-22T23:31:04.189 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:318: TEST_divergent_ec: sleep 5 2026-02-22T23:31:09.192 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: expr 2 + 2 2026-02-22T23:31:09.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:322: TEST_divergent_ec: objname=existing_4 2026-02-22T23:31:09.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:323: TEST_divergent_ec: echo 'writing non-divergent object existing_4' 2026-02-22T23:31:09.193 INFO:tasks.workunit.client.0.trial122.stdout:writing non-divergent object existing_4 2026-02-22T23:31:09.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:324: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:31:09.382 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:31:09.382 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 0 0 0 73528 0 0 14 90 14 stale+active+clean 2026-02-22T23:30:25.790872+0000 28'104 28:123 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:54:33.654791+0000 0 0 2026-02-22T23:31:09.382 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:31:09.382 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:31:09.383 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:31:09.391 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:325: TEST_divergent_ec: rados -p test put existing_4 /etc/resolv.conf 2026-02-22T23:31:09.421 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:31:09.421 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:327: TEST_divergent_ec: wait_for_clean 2026-02-22T23:31:09.421 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:31:09.421 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:31:09.421 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:31:09.422 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:31:09.422 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:31:09.422 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:31:09.422 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:31:09.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:31:09.423 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:31:09.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:31:09.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:31:09.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:31:09.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:31:09.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:31:09.493 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:31:09.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:31:09.694 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:31:09.694 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:31:09.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:31:09.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:31:09.694 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:31:09.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=150323855365 2026-02-22T23:31:09.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 150323855365 2026-02-22T23:31:09.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365' 2026-02-22T23:31:09.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:31:09.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:31:09.832 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:31:09.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:31:09.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:31:09.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:31:09.834 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:31:09.835 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:31:09.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=163208757252 2026-02-22T23:31:09.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 163208757252 2026-02-22T23:31:09.911 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-150323855365 2-163208757252' 2026-02-22T23:31:09.911 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:31:09.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-150323855365 2026-02-22T23:31:09.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:31:09.912 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:31:09.913 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-150323855365 2026-02-22T23:31:09.913 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:31:09.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=150323855365 2026-02-22T23:31:09.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 150323855365' 2026-02-22T23:31:09.915 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 150323855365 2026-02-22T23:31:09.915 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:31:10.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-22T23:31:10.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:31:10.117 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-163208757252 2026-02-22T23:31:10.117 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:31:10.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:31:10.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-163208757252 2026-02-22T23:31:10.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:31:10.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=163208757252 2026-02-22T23:31:10.121 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 163208757252 2026-02-22T23:31:10.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 163208757252' 2026-02-22T23:31:10.121 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:31:10.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 163208757252 -lt 163208757252 2026-02-22T23:31:10.331 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:31:10.331 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:10.331 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:10.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:31:10.593 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:10.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:10.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:10.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:10.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:10.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:10.594 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:10.787 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:10.788 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:10.788 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:10.788 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:11.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:11.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:31:11.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:31:11.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:31:11.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:11.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:11.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:11.145 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:11.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:11.350 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:11.350 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:11.350 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:11.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:11.596 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:11.596 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:11.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:11.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:11.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:31:11.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:11.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:31:12.048 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:12.049 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:12.049 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:12.050 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:12.050 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:12.050 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:12.050 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:12.050 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:12.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:12.247 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:12.248 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:12.248 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:12.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:12.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:12.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:12.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:12.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:12.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:12.511 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:12.511 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:12.512 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:12.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:12.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:12.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:31:12.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:12.772 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:31:13.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:13.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:13.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:13.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:13.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:13.173 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:13.174 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:13.174 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:13.373 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:13.373 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:13.374 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:13.374 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:13.640 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:13.641 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:13.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:13.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:13.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:31:13.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:13.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:31:14.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:14.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:14.708 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:14.708 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:14.708 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:14.708 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:14.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:14.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:14.901 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:14.902 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:14.902 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:14.902 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:15.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:15.156 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:15.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:15.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:15.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:31:15.412 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:15.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:31:17.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:17.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:17.014 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:17.014 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:17.014 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:17.014 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:17.014 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:17.015 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:17.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:17.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:17.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:17.208 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:17.468 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:17.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:17.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:17.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:31:17.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:17.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:31:20.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:20.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:20.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:20.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:20.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:20.928 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:20.929 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:20.929 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:21.126 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:21.127 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:21.127 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:21.128 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:21.388 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:21.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:21.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:21.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:21.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:21.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:21.389 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:21.390 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:21.390 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:21.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:21.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:21.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:31:21.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:21.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:31:28.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:28.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:28.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:28.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:28.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:28.065 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:28.066 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:28.066 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:28.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:28.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:28.266 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:28.267 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:28.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:28.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:28.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:28.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:28.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:28.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:28.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:28.536 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:28.536 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:28.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:28.806 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:28.806 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:31:28.806 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:28.806 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:31:36.106 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:36.107 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:36.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:36.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:36.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:36.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:36.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:36.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:36.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:36.308 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:36.309 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:36.309 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:36.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:36.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:36.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:36.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:36.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:36.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:36.572 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:36.572 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:36.573 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:36.840 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:36.840 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:36.841 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:31:36.841 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:31:37.095 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:31:37.095 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "2c0616d6-fe4c-4ff5-b585-ea78a46b1c2b", 2026-02-22T23:31:37.095 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:31:37.091355+0000", 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:31:37.096 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:31:37.097 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:31:37.098 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:31:37.098 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.098 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:31:37.098 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.098 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:31:37.098 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:31:37.099 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:31:37.100 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:31:37.101 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:31:37.102 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:29:57.401826Z", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:29:57.401826Z", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:31:37.103 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:31:37.104 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:31:37.105 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:31:37.106 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:31:37.107 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 41, 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:29:57.484930+0000", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:31:04.232165+0000", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:31:02.952692+0000", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:31:04.129068+0000", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:31:37.108 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 10, 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:31:37.109 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:30:24.810138+0000", 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 16389, 2026-02-22T23:31:37.110 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "type": 3, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 1, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:31:37.111 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "off", 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 1, 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 1, 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 1, 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 1, 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:31:37.112 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "28", 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:31:37.113 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "0", 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "pool", 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 1, 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 28, 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [ 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "snapid": 1, 2026-02-22T23:31:37.114 INFO:tasks.workunit.client.0.trial122.stdout: "stamp": "2026-02-22T23:30:41.189863+0000", 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "name": "snap1" 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:31:37.115 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "myprofile", 2026-02-22T23:31:37.116 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:31:37.117 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 8192, 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {} 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.118 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "4c996602-e7fe-497f-b8c0-5a182a52c84b", 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:31:37.119 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 5, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 27, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 35, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 40, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 29, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:31:37.120 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.121 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.122 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.123 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.124 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.125 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.126 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 1405372986 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/1405372986", 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/1405372986", 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/1405372986", 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/1405372986", 2026-02-22T23:31:37.127 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "b2449376-0236-402d-9cb6-140a74a80dee", 2026-02-22T23:31:37.128 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 23, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 30, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:31:37.129 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.130 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.131 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:31:37.132 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.133 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/2569739617", 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/2569739617", 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2569739617", 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2569739617", 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.134 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32", 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 14, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 27, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 38, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 31, 2026-02-22T23:31:37.135 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.136 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.137 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.138 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 43963801 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.139 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/43963801", 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/43963801", 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/43963801", 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/43963801", 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.140 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:30:51.849687+0000", 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:30:06.427011+0000", 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:30:52.130049+0000", 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:31:37.141 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:30:14.486835+0000", 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:30:52.408193+0000", 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:30:22.820066+0000", 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.142 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "myprofile": { 2026-02-22T23:31:37.143 INFO:tasks.workunit.client.0.trial122.stdout: "crush-device-class": "", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "crush-failure-domain": "osd", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "crush-num-failure-domains": "0", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "crush-osds-per-failure-domain": "0", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "crush-root": "default", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "jerasure-per-chunk-alignment": "false", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "m": "1", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van", 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: "w": "8" 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.144 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.145 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/1405372986,v1:127.0.0.1:6805/1405372986]", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:31:37.146 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:30:00.144681Z", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:31:37.147 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/1405372986,v1:127.0.0.1:6803/1405372986]", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/1405372986,v1:127.0.0.1:6809/1405372986]", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/1405372986,v1:127.0.0.1:6807/1405372986]", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:31:37.148 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/2569739617,v1:127.0.0.1:6813/2569739617]", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:31:37.149 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:30:08.143217Z", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/2569739617,v1:127.0.0.1:6811/2569739617]", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2569739617,v1:127.0.0.1:6817/2569739617]", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2569739617,v1:127.0.0.1:6815/2569739617]", 2026-02-22T23:31:37.150 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/43963801,v1:127.0.0.1:6813/43963801]", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:31:37.151 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:30:16.154117Z", 2026-02-22T23:31:37.152 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/43963801,v1:127.0.0.1:6811/43963801]", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/43963801,v1:127.0.0.1:6817/43963801]", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/43963801,v1:127.0.0.1:6815/43963801]", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:31:37.153 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 0, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [] 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 35 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 38 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 41, 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:31:37.154 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:31:37.155 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:31:37.156 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:31:37.157 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:31:37.158 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.159 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.160 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.161 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 1, 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "test", 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "type": 3, 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "op": "set_chooseleaf_tries", 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "num": 5 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "op": "set_choose_tries", 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "num": 100 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.162 INFO:tasks.workunit.client.0.trial122.stderr:report 4168335320 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_indep", 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:31:37.163 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:29:57:484719+0000", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:31:37.164 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 1, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73767, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 101, 2026-02-22T23:31:37.165 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 1, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 303, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 101, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 101, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:31:37.166 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 18, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 18, 2026-02-22T23:31:37.167 INFO:tasks.workunit.client.0.trial122.stdout: "up": 3, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 3, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83896, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2116, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488904, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036637696, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2166784, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 1538924, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679083 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:31:37.168 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 83896, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2116, 2026-02-22T23:31:37.169 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 10, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 81717, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314488904, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322036637696, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2166784, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 1538924, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 10389, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 83679083 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.170 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73767, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 101, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 1, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 303, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 101, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 101, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:31:37.171 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1241088, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 1241088, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 18, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 18, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "up": 3, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 3, 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.172 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 150323855370 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705680 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 163208757257 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num": 1 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2147483647, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 1, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:31:37.173 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 4, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:31:15.952798+0000", 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "services": { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "osd": { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "daemons": { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "summary": "", 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "0": { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "2": { 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:31:37.174 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 47, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 242, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 340, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:31:37.175 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.0 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:31:37.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:31:37.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:31:37.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 0 --op log --pgid 1.0 2026-02-22T23:31:37.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:31:37.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:31:37.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=0 2026-02-22T23:31:37.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:31:37.210 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/0 2026-02-22T23:31:37.211 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/0 --op log --pgid 1.0 2026-02-22T23:31:37.854 INFO:tasks.workunit.client.0.trial122.stderr:PG '1.0' not found 2026-02-22T23:31:38.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-22T23:31:38.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 0 2026-02-22T23:31:38.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:31:38.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/0 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:31:38.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/0' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/0/journal' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:31:38.398 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:31:38.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:31:38.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:31:38.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:31:38.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:31:38.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:31:38.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/0 2026-02-22T23:31:38.400 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:31:38.401 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:31:38.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/0 --osd-journal=td/divergent-priors/0/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:31:38.402 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/0/whoami 2026-02-22T23:31:38.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:31:38.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:31:38.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:31:38.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:31:38.417 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:38.414+0000 7f6f282ed8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:31:38.418 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:38.414+0000 7f6f282ed8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:31:38.419 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:38.418+0000 7f6f282ed8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:31:38.594 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:31:38.595 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:31:38.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:31:38.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:31:38.797 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 35 up_thru 40 down_at 29 last_clean_interval [5,27) [v2:127.0.0.1:6802/1405372986,v1:127.0.0.1:6803/1405372986] [v2:127.0.0.1:6804/1405372986,v1:127.0.0.1:6805/1405372986] exists,up 4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:31:38.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:31:38.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:31:38.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:31:38.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:330: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:31:38.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:332: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.2 2026-02-22T23:31:38.800 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:31:38.800 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:31:38.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:31:38.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:31:38.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:31:38.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:31:38.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:333: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 2 --op log --pgid 1.0 2026-02-22T23:31:38.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:31:38.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:31:38.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=2 2026-02-22T23:31:38.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:31:38.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/2 2026-02-22T23:31:38.907 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/2 --op log --pgid 1.0 2026-02-22T23:31:39.544 INFO:tasks.workunit.client.0.trial122.stderr:PG '1.0' not found 2026-02-22T23:31:39.692 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:39.690+0000 7f6f282ed8c0 -1 Falling back to public interface 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:334: TEST_divergent_ec: activate_osd td/divergent-priors 2 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/2 2026-02-22T23:31:40.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/2' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/2/journal' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:31:40.081 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:31:40.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:31:40.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/2 2026-02-22T23:31:40.083 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:31:40.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:31:40.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/2 --osd-journal=td/divergent-priors/2/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:31:40.084 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/2/whoami 2026-02-22T23:31:40.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:31:40.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:31:40.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:31:40.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:31:40.097 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:40.094+0000 7f0d2db3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:31:40.098 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:40.098+0000 7f0d2db3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:31:40.100 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:40.098+0000 7f0d2db3e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:31:40.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:31:40.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:31:40.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:31:40.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:31:40.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:31:40.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:31:40.289 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:31:40.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:31:40.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:31:40.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:31:40.493 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 38 up_thru 0 down_at 31 last_clean_interval [14,27) [v2:127.0.0.1:6810/43963801,v1:127.0.0.1:6811/43963801] [v2:127.0.0.1:6812/43963801,v1:127.0.0.1:6813/43963801] exists,up fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:336: TEST_divergent_ec: _objectstore_tool_nodown td/divergent-priors 1 --op log --pgid 1.0 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1257: _objectstore_tool_nodown: local dir=td/divergent-priors 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1258: _objectstore_tool_nodown: shift 2026-02-22T23:31:40.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1259: _objectstore_tool_nodown: local id=1 2026-02-22T23:31:40.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nodown: shift 2026-02-22T23:31:40.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nodown: local osd_data=td/divergent-priors/1 2026-02-22T23:31:40.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1264: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/divergent-priors/1 --op log --pgid 1.0 2026-02-22T23:31:41.172 INFO:tasks.workunit.client.0.trial122.stderr:PG '1.0' not found 2026-02-22T23:31:41.364 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:41.362+0000 7f0d2db3e8c0 -1 Falling back to public interface 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nodown: return 1 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: WAIT_FOR_CLEAN_TIMEOUT=20 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:338: TEST_divergent_ec: wait_for_clean 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:31:41.713 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 20 .1 2026-02-22T23:31:41.714 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:31:41.714 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:31:41.714 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:31:41.714 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:31:41.714 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:31:41.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '7.3') 2026-02-22T23:31:41.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:31:41.778 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:31:41.779 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:31:41.779 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:31:41.779 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:31:41.986 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:31:41.987 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:31:41.987 INFO:tasks.workunit.client.0.trial122.stderr:2' 2026-02-22T23:31:41.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:31:41.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:31:41.987 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:31:42.728 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:42.726+0000 7f6f282ed8c0 -1 osd.0 41 log_to_monitors true 2026-02-22T23:31:44.334 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:31:44.330+0000 7f0d2db3e8c0 -1 osd.2 41 log_to_monitors true 2026-02-22T23:31:47.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=188978561027 2026-02-22T23:31:47.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 188978561027 2026-02-22T23:31:47.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561027' 2026-02-22T23:31:47.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:31:47.076 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:31:47.137 INFO:tasks.workunit.client.0.trial122.stderr:Error ENXIO: problem getting command descriptions from osd.1 2026-02-22T23:31:47.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq= 2026-02-22T23:31:47.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z '' 2026-02-22T23:31:47.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: continue 2026-02-22T23:31:47.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:31:47.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:31:47.220 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=206158430211 2026-02-22T23:31:47.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 206158430211 2026-02-22T23:31:47.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-188978561027 2-206158430211' 2026-02-22T23:31:47.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:31:47.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-188978561027 2026-02-22T23:31:47.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:31:47.222 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:31:47.223 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-188978561027 2026-02-22T23:31:47.223 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:31:47.224 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=188978561027 2026-02-22T23:31:47.225 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 188978561027 2026-02-22T23:31:47.225 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 188978561027' 2026-02-22T23:31:47.225 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:31:47.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 188978561027 2026-02-22T23:31:47.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:31:48.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:31:48.440 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:31:48.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 188978561028 -lt 188978561027 2026-02-22T23:31:48.642 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:31:48.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-206158430211 2026-02-22T23:31:48.643 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:31:48.644 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:31:48.645 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-206158430211 2026-02-22T23:31:48.645 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:31:48.647 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=206158430211 2026-02-22T23:31:48.647 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 206158430211 2026-02-22T23:31:48.647 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 206158430211' 2026-02-22T23:31:48.647 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:31:48.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 206158430211 -lt 206158430211 2026-02-22T23:31:48.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:31:48.849 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:48.849 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:49.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:31:49.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:49.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:49.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:49.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:49.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:49.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:49.115 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:49.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:49.312 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:49.312 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:49.313 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:49.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:49.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' -1 2026-02-22T23:31:49.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1674: wait_for_clean: loop=0 2026-02-22T23:31:49.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1675: wait_for_clean: num_active_clean=0 2026-02-22T23:31:49.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:49.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.1 2026-02-22T23:31:49.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:49.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:49.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:49.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:49.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:49.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:49.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:49.673 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:49.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:49.865 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:49.865 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:49.865 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:50.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:50.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:50.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:50.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:50.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:50.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:50.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:50.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:50.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:50.373 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:50.374 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:50.374 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 1 >= 8 )) 2026-02-22T23:31:50.374 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:50.374 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.2 2026-02-22T23:31:50.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:50.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:50.575 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:50.575 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:50.575 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:50.575 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:50.576 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:50.576 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:50.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:50.769 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:50.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:50.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:51.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:51.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:51.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:51.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:51.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:51.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:51.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:51.024 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:51.024 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:51.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:51.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:51.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 2 >= 8 )) 2026-02-22T23:31:51.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:51.291 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.4 2026-02-22T23:31:51.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:51.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:51.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:51.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:51.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:51.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:51.693 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:51.693 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:51.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:51.896 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:51.896 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:51.896 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:52.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:52.158 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:52.158 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:52.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:52.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:52.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 3 >= 8 )) 2026-02-22T23:31:52.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:52.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 0.8 2026-02-22T23:31:53.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:53.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:53.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:53.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:53.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:53.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:53.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:53.222 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:53.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:53.420 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:53.420 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:53.420 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:53.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:53.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:53.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:53.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:53.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:53.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:53.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:53.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:53.674 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 4 >= 8 )) 2026-02-22T23:31:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:53.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 1.6 2026-02-22T23:31:55.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:55.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:55.542 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:55.542 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:55.542 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:55.542 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:55.542 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:55.543 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:55.738 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:55.738 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:55.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:55.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:56.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:56.004 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:31:56.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:31:56.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:31:56.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 5 >= 8 )) 2026-02-22T23:31:56.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:31:56.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 3.2 2026-02-22T23:31:59.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:31:59.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:31:59.468 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:31:59.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:31:59.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:31:59.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:31:59.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:31:59.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:31:59.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:31:59.668 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:31:59.669 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:31:59.669 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:31:59.926 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:31:59.926 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:31:59.927 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:32:00.190 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:32:00.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:32:00.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 6 >= 8 )) 2026-02-22T23:32:00.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:32:00.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 6.4 2026-02-22T23:32:06.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:32:06.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:32:06.592 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:32:06.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:32:06.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:32:06.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:32:06.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:32:06.593 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:32:06.791 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:32:06.792 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:32:06.792 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:32:06.793 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:32:07.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:32:07.048 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:32:07.048 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:32:07.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:32:07.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:32:07.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 7 >= 8 )) 2026-02-22T23:32:07.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1683: wait_for_clean: eval 2026-02-22T23:32:07.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1684: wait_for_clean: sleep 7.3 2026-02-22T23:32:14.609 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1685: wait_for_clean: loop+=1 2026-02-22T23:32:14.609 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:32:14.610 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:32:14.611 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:32:14.611 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:32:14.611 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:32:14.611 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:32:14.611 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:32:14.805 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=0 2026-02-22T23:32:14.806 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:32:14.806 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:32:14.806 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 0 = 1 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1673: wait_for_clean: test 0 '!=' 0 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1676: wait_for_clean: get_is_making_recovery_progress 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1334: get_is_making_recovery_progress: local recovery_progress 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1335: get_is_making_recovery_progress: recovery_progress+='.recovering_keys_per_sec + ' 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1336: get_is_making_recovery_progress: recovery_progress+='.recovering_bytes_per_sec + ' 2026-02-22T23:32:15.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1337: get_is_making_recovery_progress: recovery_progress+=.recovering_objects_per_sec 2026-02-22T23:32:15.071 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: ceph --format json status 2026-02-22T23:32:15.071 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: jq -r '.pgmap | .recovering_keys_per_sec + .recovering_bytes_per_sec + .recovering_objects_per_sec' 2026-02-22T23:32:15.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1339: get_is_making_recovery_progress: local progress=null 2026-02-22T23:32:15.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1340: get_is_making_recovery_progress: test null '!=' null 2026-02-22T23:32:15.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: (( 8 >= 8 )) 2026-02-22T23:32:15.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1679: wait_for_clean: ceph report 2026-02-22T23:32:15.592 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:32:15.592 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_fingerprint": "2c0616d6-fe4c-4ff5-b585-ea78a46b1c2b", 2026-02-22T23:32:15.592 INFO:tasks.workunit.client.0.trial122.stdout: "version": "19.2.3-689-gf90f4eae", 2026-02-22T23:32:15.592 INFO:tasks.workunit.client.0.trial122.stdout: "commit": "f90f4eaebb28b619cf76e80cbbe997656da69579", 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "timestamp": "2026-02-22T23:32:15.592427+0000", 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "tag": "", 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "health": { 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "status": "HEALTH_WARN", 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "checks": { 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "OSDMAP_FLAGS": { 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "message": "nodown,noout,noin flag(s) set", 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: "count": 17 2026-02-22T23:32:15.593 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [], 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "OSD_DOWN": { 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 osds down", 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: "message": "osd.1 (root=default,host=trial122) is down" 2026-02-22T23:32:15.594 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "PG_DEGRADED": { 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "message": "Degraded data redundancy: 101/303 objects degraded (33.333%), 1 pg degraded", 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:32:15.595 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "message": "pg 1.0 is active+undersized+degraded, acting [NONE,0,2]" 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "POOL_APP_NOT_ENABLED": { 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "severity": "HEALTH_WARN", 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "summary": { 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "message": "1 pool(s) do not have an application enabled", 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "count": 1 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.596 INFO:tasks.workunit.client.0.trial122.stdout: "detail": [ 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: "message": "application not enabled on pool 'test'" 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: "message": "use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications." 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: "muted": false 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: "mutes": [] 2026-02-22T23:32:15.597 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_first_committed": 1, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "monmap_last_committed": 1, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "monmap": { 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:29:57.401826Z", 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:29:57.401826Z", 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release": 19, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "min_mon_release_name": "squid", 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "election_strategy": 1, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "disallowed_leaders": "", 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": false, 2026-02-22T23:32:15.598 INFO:tasks.workunit.client.0.trial122.stdout: "tiebreaker_mon": "", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "removed_ranks": "", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "features": { 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "persistent": [ 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "kraken", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "luminous", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "mimic", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap-prune", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "nautilus", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "octopus", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "pacific", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "elector-pinging", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "quincy", 2026-02-22T23:32:15.599 INFO:tasks.workunit.client.0.trial122.stdout: "reef", 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "squid" 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "optional": [] 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "mons": [ 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "rank": 0, 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "name": "a", 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.600 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115", 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 0 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:7115/0", 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:7115/0", 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "priority": 0, 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 0, 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: "crush_location": "{}" 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.601 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "quorum": [ 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: 0 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap": { 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 49, 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "fsid": "e462d5e6-2932-48de-9d0e-5c8a75d6733d", 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "created": "2026-02-22T23:29:57.484930+0000", 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:31:44.720601+0000", 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "last_up_change": "2026-02-22T23:31:44.618208+0000", 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "last_in_change": "2026-02-22T23:31:04.129068+0000", 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "flags": "nodown,noout,noin,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2026-02-22T23:32:15.602 INFO:tasks.workunit.client.0.trial122.stdout: "flags_num": 5800384, 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "flags_set": [ 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "nodown", 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "noin", 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "noout", 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "pglog_hardlimit", 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snapdirs", 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "recovery_deletes", 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "sortbitwise" 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "crush_version": 12, 2026-02-22T23:32:15.603 INFO:tasks.workunit.client.0.trial122.stdout: "full_ratio": 0.99000000953674316, 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "backfillfull_ratio": 0.99000000953674316, 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "nearfull_ratio": 0.85000002384185791, 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_snapshot": "", 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "pool_max": 1, 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "max_osd": 3, 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "require_min_compat_client": "luminous", 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "min_compat_client": "jewel", 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "require_osd_release": "squid", 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "allow_crimson": false, 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "pools": [ 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.604 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "pool_name": "test", 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "create_time": "2026-02-22T23:30:24.810138+0000", 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "flags": 16389, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "flags_names": "hashpspool,ec_overwrites,pool_snaps", 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "type": 3, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "size": 3, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "min_size": 2, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "crush_rule": 1, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_count": 0, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_target": 0, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_barrier": 0, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2026-02-22T23:32:15.605 INFO:tasks.workunit.client.0.trial122.stdout: "is_stretch_pool": false, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "object_hash": 2, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "pg_autoscale_mode": "off", 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num": 1, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num": 1, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "pg_placement_num_target": 1, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_target": 1, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "pg_num_pending": 1, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "last_pg_merge_meta": { 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "source_pgid": "0.0", 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "ready_epoch": 0, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_started": 0, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": 0, 2026-02-22T23:32:15.606 INFO:tasks.workunit.client.0.trial122.stdout: "source_version": "0'0", 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "target_version": "0'0" 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "last_change": "28", 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend": "0", 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_prenautilus": "0", 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "last_force_op_resend_preluminous": "0", 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "auid": 0, 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "snap_mode": "pool", 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "snap_seq": 1, 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "snap_epoch": 28, 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "pool_snaps": [ 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "snapid": 1, 2026-02-22T23:32:15.607 INFO:tasks.workunit.client.0.trial122.stdout: "stamp": "2026-02-22T23:30:41.189863+0000", 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "name": "snap1" 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps": "[]", 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_bytes": 0, 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "quota_max_objects": 0, 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "tiers": [], 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "tier_of": -1, 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "read_tier": -1, 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "write_tier": -1, 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "cache_mode": "none", 2026-02-22T23:32:15.608 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_bytes": 0, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "target_max_objects": 0, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_ratio_micro": 400000, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "cache_target_full_ratio_micro": 800000, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_flush_age": 0, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "cache_min_evict_age": 0, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profile": "myprofile", 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_params": { 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "type": "none" 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_period": 0, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_count": 0, 2026-02-22T23:32:15.609 INFO:tasks.workunit.client.0.trial122.stdout: "use_gmt_hitset": true, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "min_read_recency_for_promote": 0, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "min_write_recency_for_promote": 0, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_grade_decay_rate": 0, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "hit_set_search_last_n": 0, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "grade_table": [], 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "stripe_width": 8192, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "expected_num_objects": 0, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "fast_read": false, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "options": {}, 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: "application_metadata": {} 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.610 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "osds": [ 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "4c996602-e7fe-497f-b8c0-5a182a52c84b", 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 35, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 41, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 44, 2026-02-22T23:32:15.611 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 48, 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 43, 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6802", 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6803", 2026-02-22T23:32:15.612 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6804", 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.613 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6805", 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6808", 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.614 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6809", 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6806", 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.615 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6807", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 112363510 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6803/112363510", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6805/112363510", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6809/112363510", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6807/112363510", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:32:15.616 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "b2449376-0236-402d-9cb6-140a74a80dee", 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "up": 0, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 0, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 0, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 9, 2026-02-22T23:32:15.617 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 23, 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 30, 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.618 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.619 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.620 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.621 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 2569739617 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/2569739617", 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/2569739617", 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/2569739617", 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/2569739617", 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: "exists" 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.622 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "uuid": "fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32", 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "up": 1, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "in": 1, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 1, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "primary_affinity": 1, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_begin": 38, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "last_clean_end": 41, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 48, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "up_thru": 0, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "down_at": 47, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "lost_at": 0, 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "public_addrs": { 2026-02-22T23:32:15.623 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6810", 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6811", 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addrs": { 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.624 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6812", 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6813", 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addrs": { 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.625 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6816", 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6817", 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addrs": { 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "addrvec": [ 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.626 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v2", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6814", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "type": "v1", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "127.0.0.1:6815", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "nonce": 3506527952 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "public_addr": "127.0.0.1:6811/3506527952", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "cluster_addr": "127.0.0.1:6813/3506527952", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_back_addr": "127.0.0.1:6817/3506527952", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "heartbeat_front_addr": "127.0.0.1:6815/3506527952", 2026-02-22T23:32:15.627 INFO:tasks.workunit.client.0.trial122.stdout: "state": [ 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "exists", 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "up" 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "osd_xinfo": [ 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:31:42.882296+0000", 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:32:15.628 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:30:06.427011+0000", 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:30:52.130049+0000", 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:30:14.486835+0000", 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.629 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "down_stamp": "2026-02-22T23:31:44.516277+0000", 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_probability": 0, 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "laggy_interval": 0, 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "features": 4540701547738038271, 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "old_weight": 0, 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "last_purged_snaps_scrub": "2026-02-22T23:30:22.820066+0000", 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "dead_epoch": 0 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap": [], 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_items": [], 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "pg_upmap_primaries": [], 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "pg_temp": [], 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "primary_temp": [], 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "blocklist": {}, 2026-02-22T23:32:15.630 INFO:tasks.workunit.client.0.trial122.stdout: "range_blocklist": {}, 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "erasure_code_profiles": { 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "default": { 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "m": "2", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van" 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "myprofile": { 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "crush-device-class": "", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "crush-failure-domain": "osd", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "crush-num-failure-domains": "0", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "crush-osds-per-failure-domain": "0", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "crush-root": "default", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "jerasure-per-chunk-alignment": "false", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "k": "2", 2026-02-22T23:32:15.631 INFO:tasks.workunit.client.0.trial122.stdout: "m": "1", 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "plugin": "jerasure", 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "technique": "reed_sol_van", 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "w": "8" 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "removed_snaps_queue": [], 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "new_removed_snaps": [], 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "new_purged_snaps": [], 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "crush_node_flags": {}, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "device_class_flags": {}, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode": { 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_enabled": false, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_bucket_count": 0, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "degraded_stretch_mode": 0, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "recovering_stretch_mode": 0, 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: "stretch_mode_bucket": 0 2026-02-22T23:32:15.632 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "osd_metadata": [ 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6804/112363510,v1:127.0.0.1:6805/112363510]", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:32:15.633 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/0/block", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:30:00.144681Z", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:32:15.634 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6802/112363510,v1:127.0.0.1:6803/112363510]", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6808/112363510,v1:127.0.0.1:6809/112363510]", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6806/112363510,v1:127.0.0.1:6807/112363510]", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/0", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:32:15.635 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/2569739617,v1:127.0.0.1:6813/2569739617]", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/1/block", 2026-02-22T23:32:15.636 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:30:08.143217Z", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/2569739617,v1:127.0.0.1:6811/2569739617]", 2026-02-22T23:32:15.637 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/2569739617,v1:127.0.0.1:6817/2569739617]", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/2569739617,v1:127.0.0.1:6815/2569739617]", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/1", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "arch": "x86_64", 2026-02-22T23:32:15.638 INFO:tasks.workunit.client.0.trial122.stdout: "back_addr": "[v2:127.0.0.1:6812/3506527952,v1:127.0.0.1:6813/3506527952]", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "back_iface": "", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs": "1", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_db": "0", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_dedicated_wal": "0", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluefs_single_shared_device": "1", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocation_from_file": "1", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_allocator": "hybrid", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_access_mode": "file", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_block_size": "4096", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_devices": "nvme0n1", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_driver": "KernelDevice", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_optimal_io_size": "4096", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_path": "td/divergent-priors/2/block", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_rotational": "0", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_size": "107374182400", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_support_discard": "1", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_bdev_type": "ssd", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "bluestore_min_alloc_size": "4096", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_release": "squid", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:32:15.639 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_short": "19.2.3-689-gf90f4eae", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "ceph_version_when_created": "ceph version 19.2.3-689-gf90f4eae (f90f4eaebb28b619cf76e80cbbe997656da69579) squid (stable)", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "cpu": "AMD EPYC 4464P 12-Core Processor", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "created_at": "2026-02-22T23:30:16.154117Z", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "default_device_class": "ssd", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "device_ids": "nvme0n1=Micron_7450_MTFDKBA960TFR_25174FCD6F09", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "device_paths": "nvme0n1=/dev/disk/by-path/pci-0000:02:00.0-nvme-1", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "devices": "nvme0n1", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "distro": "ubuntu", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "distro_description": "Ubuntu 22.04.5 LTS", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "distro_version": "22.04", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "front_addr": "[v2:127.0.0.1:6810/3506527952,v1:127.0.0.1:6811/3506527952]", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "front_iface": "", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "hb_back_addr": "[v2:127.0.0.1:6816/3506527952,v1:127.0.0.1:6817/3506527952]", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "hb_front_addr": "[v2:127.0.0.1:6814/3506527952,v1:127.0.0.1:6815/3506527952]", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "hostname": "trial122", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "journal_rotational": "0", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_description": "#180-Ubuntu SMP Fri Jan 9 16:10:31 UTC 2026", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "kernel_version": "5.15.0-170-generic", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "mem_swap_kb": "8388604", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "mem_total_kb": "130961700", 2026-02-22T23:32:15.640 INFO:tasks.workunit.client.0.trial122.stdout: "network_numa_unknown_ifaces": "back_iface,front_iface", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_node": "0", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "objectstore_numa_nodes": "0", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "os": "Linux", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "osd_data": "td/divergent-priors/2", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "osd_objectstore": "bluestore", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "osdspec_affinity": "", 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "rotational": "0" 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_clean_epochs": { 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "min_last_epoch_clean": 24, 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "last_epoch_clean": { 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "per_pool": [ 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "floor": 24 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "osd_epochs": [ 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.641 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 45 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 49 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_first_committed": 1, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "osdmap_last_committed": 49, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "crushmap": { 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "devices": [ 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.0", 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.1", 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:32:15.642 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd.2", 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "class": "ssd" 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "types": [ 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 0, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "name": "osd" 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "name": "host" 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 2, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "name": "chassis" 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 3, 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: "name": "rack" 2026-02-22T23:32:15.643 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 4, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "name": "row" 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 5, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pdu" 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 6, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "name": "pod" 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 7, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "name": "room" 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 8, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "name": "datacenter" 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.644 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 9, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "name": "zone" 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 10, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "name": "region" 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "name": "root" 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "buckets": [ 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "id": -1, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default", 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.645 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "id": -2, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "name": "default~ssd", 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 11, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "root", 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "id": -3, 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122", 2026-02-22T23:32:15.646 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.647 INFO:tasks.workunit.client.0.trial122.stdout: "id": -4, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "name": "trial122~ssd", 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "type_id": 1, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "type_name": "host", 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 19206, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "alg": "straw2", 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "hash": "rjenkins1", 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "items": [ 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "id": 0, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 0 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "id": 1, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 1 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "id": 2, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "weight": 6402, 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "pos": 2 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: "rules": [ 2026-02-22T23:32:15.648 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 0, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "replicated_rule", 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "type": 1, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_firstn", 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "rule_id": 1, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "rule_name": "test", 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "type": 3, 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "steps": [ 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.649 INFO:tasks.workunit.client.0.trial122.stdout: "op": "set_chooseleaf_tries", 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "num": 5 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "op": "set_choose_tries", 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "num": 100 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "op": "take", 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stderr:report 449883153 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "item": -1, 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "item_name": "default" 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.650 INFO:tasks.workunit.client.0.trial122.stdout: "op": "choose_indep", 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "num": 0, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "type": "osd" 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "op": "emit" 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: ] 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "tunables": { 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_tries": 0, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "choose_local_fallback_tries": 0, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "choose_total_tries": 50, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_descend_once": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_vary_r": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "chooseleaf_stable": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "msr_descents": 100, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "msr_collision_tries": 100, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "straw_calc_version": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "allowed_bucket_algs": 54, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "profile": "jewel", 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "optimal_tunables": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "legacy_tunables": 0, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "minimum_required_version": "jewel", 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables2": 1, 2026-02-22T23:32:15.651 INFO:tasks.workunit.client.0.trial122.stdout: "has_v2_rules": 1, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables3": 1, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "has_v3_rules": 0, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "has_v4_buckets": 1, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "require_feature_tunables5": 1, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "has_v5_rules": 0, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "has_msr_rules": 0 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "choose_args": {} 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "fsmap": { 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 1, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "btime": "2026-02-22T23:29:57:484719+0000", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "default_fscid": -1, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "compat": { 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "compat": {}, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "ro_compat": {}, 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "incompat": { 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_1": "base v0.20", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_2": "client writeable ranges", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_3": "default file layouts on dirs", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_4": "dir inode in separate object", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_5": "mds uses versioned encoding", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_6": "dirfrag is stored in omap", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_8": "no anchor table", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_9": "file layout v2", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_10": "snaprealm v2", 2026-02-22T23:32:15.652 INFO:tasks.workunit.client.0.trial122.stdout: "feature_11": "minor log segments", 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "feature_12": "quiesce subvolumes" 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "feature_flags": { 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "enable_multiple": true, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "ever_enabled_multiple": true 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "standbys": [], 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "filesystems": [] 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_first_committed": 1, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "mdsmap_last_committed": 1, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "auth": { 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 5, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_secrets": 3 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_active": 1, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_unknown": 0, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_osd": 3, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "pool_sum": { 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73767, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 101, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 1, 2026-02-22T23:32:15.653 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 303, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 101, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 101, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:32:15.654 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 18, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 18, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "up": 3, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 3, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 0 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum": { 2026-02-22T23:32:15.655 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 85080, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2244, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 15, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 82736, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314487720, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322035425280, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2297856, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 1593792, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 16248, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 84721800 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:32:15.656 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "osd_sum_by_class": { 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "ssd": { 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "up_from": 0, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 0, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "num_pgs": 3, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "num_osds": 3, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_osds": 3, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "num_per_pool_omap_osds": 3, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "kb": 314572800, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used": 85080, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_data": 2244, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_omap": 15, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "kb_used_meta": 82736, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "kb_avail": 314487720, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "statfs": { 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "total": 322122547200, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "available": 322035425280, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 2297856, 2026-02-22T23:32:15.657 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 1593792, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 16248, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 84721800 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "hb_peers": [], 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "snap_trim_queue_len": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "num_snap_trimming": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "num_shards_repaired": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "op_queue_age_hist": { 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "histogram": [], 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "upper_bound": 1 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "perf_stat": { 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ms": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ms": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "commit_latency_ns": 0, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "apply_latency_ns": 0 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "alerts": [], 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "network_ping_times": [] 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "pool_stats": [ 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "poolid": 1, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg": 1, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "stat_sum": { 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes": 73767, 2026-02-22T23:32:15.658 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects": 101, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_clones": 1, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_object_copies": 303, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing_on_primary": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_missing": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_degraded": 101, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_misplaced": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_unfound": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_dirty": 101, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_whiteouts": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_read": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_read_kb": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_write": 101, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_write_kb": 101, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_scrub_errors": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_shallow_scrub_errors": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_deep_scrub_errors": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_recovered": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_recovered": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_keys_recovered": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_omap": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_hit_set_archive": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_bytes_hit_set_archive": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_kb": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_kb": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_promote": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_high": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_flush_mode_low": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_some": 0, 2026-02-22T23:32:15.659 INFO:tasks.workunit.client.0.trial122.stdout: "num_evict_mode_full": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_pinned": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_legacy_snapsets": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_large_omap_objects": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_manifest": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_bytes": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_omap_keys": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_objects_repaired": 0 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "store_stats": { 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "total": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "available": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "internally_reserved": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "allocated": 1241088, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "data_stored": 1241088, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_allocated": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "data_compressed_original": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "omap_allocated": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "internal_metadata": 0 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "log_size": 18, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "ondisk_log_size": 18, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "up": 3, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "acting": 3, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "num_store_stats": 3 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "osd_stats": [ 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 188978561033 2026-02-22T23:32:15.660 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 38654705680 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "seq": 206158430216 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_state": [ 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "state": "active+undersized+degraded", 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num": 1 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_pg_by_osd": [ 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2147483647, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 2, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 0, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 1, 2026-02-22T23:32:15.661 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 1, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "osd": 1, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "num_primary_pg": 0, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "num_acting_pg": 0, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "num_up_not_acting_pg": 0 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": [ 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "pool": 1, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "purged_snaps": {} 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: ], 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "servicemap": { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "epoch": 4, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "modified": "2026-02-22T23:31:15.952798+0000", 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "services": { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "osd": { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "daemons": { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "summary": "", 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "0": { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "2": { 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "start_epoch": 0, 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "start_stamp": "0.000000", 2026-02-22T23:32:15.662 INFO:tasks.workunit.client.0.trial122.stdout: "gid": 0, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "addr": "(unrecognized address family 0)/0", 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "metadata": {}, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "task_status": {} 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: }, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_first_committed": 1, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "mgrstat_last_committed": 66, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "logm_first_committed": 1, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "logm_last_committed": 323, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "paxos": { 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "first_committed": 1, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "last_committed": 448, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "last_pn": 0, 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: "accepted_pn": 0 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout: } 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1680: wait_for_clean: return 1 2026-02-22T23:32:15.663 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery 2026-02-22T23:32:15.664 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:341: TEST_divergent_ec: echo 'delay recovery' 2026-02-22T23:32:15.664 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:342: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:15.808 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:15.808 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 49:139 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 no scrub is scheduled 0 0 2026-02-22T23:32:15.808 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:15.808 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:15.808 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:15.817 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:32:15.818 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.0 2026-02-22T23:32:15.818 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.0 2026-02-22T23:32:15.818 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.0 ']' 2026-02-22T23:32:15.818 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:32:15.819 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:15.819 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:15.819 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.0.asok 2026-02-22T23:32:15.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-22T23:32:15.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.0.asok set_recovery_delay 100000 2026-02-22T23:32:15.875 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:32:15.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:343: TEST_divergent_ec: for i in $non_divergent 2026-02-22T23:32:15.882 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: get_asok_path osd.2 2026-02-22T23:32:15.882 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.2 2026-02-22T23:32:15.882 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.2 ']' 2026-02-22T23:32:15.883 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:32:15.883 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:15.883 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:15.883 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.2.asok 2026-02-22T23:32:15.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: CEPH_ARGS= 2026-02-22T23:32:15.883 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:345: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.2.asok set_recovery_delay 100000 2026-02-22T23:32:15.938 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:32:15.944 INFO:tasks.workunit.client.0.trial122.stdout:revive divergent 1 2026-02-22T23:32:15.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:349: TEST_divergent_ec: echo 'revive divergent 1' 2026-02-22T23:32:15.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:350: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:16.140 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:16.140 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 49:139 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 no scrub is scheduled 0 0 2026-02-22T23:32:16.140 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:16.140 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:16.140 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:16.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:351: TEST_divergent_ec: ceph osd set noup 2026-02-22T23:32:16.466 INFO:tasks.workunit.client.0.trial122.stderr:noup is set 2026-02-22T23:32:16.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:352: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-22T23:32:16.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:32:16.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:32:16.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:32:16.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:32:16.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:32:16.483 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:32:16.484 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:32:16.502 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:16.497+0000 7f7515ee48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:16.504 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:16.501+0000 7f7515ee48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:16.505 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:16.501+0000 7f7515ee48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:16.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:880: activate_osd: return 0 2026-02-22T23:32:16.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:353: TEST_divergent_ec: sleep 5 2026-02-22T23:32:17.732 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:17.729+0000 7f7515ee48c0 -1 Falling back to public interface 2026-02-22T23:32:20.702 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:20.697+0000 7f7515ee48c0 -1 osd.1 28 log_to_monitors true 2026-02-22T23:32:21.683 INFO:tasks.workunit.client.0.trial122.stdout:delay recovery divergent 2026-02-22T23:32:21.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:355: TEST_divergent_ec: echo 'delay recovery divergent' 2026-02-22T23:32:21.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:356: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:21.865 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:21.865 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 51:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:13:05.082410+0000 0 0 2026-02-22T23:32:21.865 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:21.865 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:21.866 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:21.874 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: get_asok_path osd.1 2026-02-22T23:32:21.874 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=osd.1 2026-02-22T23:32:21.874 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n osd.1 ']' 2026-02-22T23:32:21.874 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:32:21.875 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:21.875 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:21.875 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.20359/ceph-osd.1.asok 2026-02-22T23:32:21.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: CEPH_ARGS= 2026-02-22T23:32:21.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:357: TEST_divergent_ec: ceph --admin-daemon /tmp/ceph-asok.20359/ceph-osd.1.asok set_recovery_delay 100000 2026-02-22T23:32:21.929 INFO:tasks.workunit.client.0.trial122.stdout:set_recovery_delay: set osd_recovery_delay_start to 100000 2026-02-22T23:32:21.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:359: TEST_divergent_ec: ceph osd unset noup 2026-02-22T23:32:22.213 INFO:tasks.workunit.client.0.trial122.stderr:noup is unset 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:361: TEST_divergent_ec: wait_for_osd up 0 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:22.227 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:32:22.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:32:22.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:22.228 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:32:22.422 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 44 up_thru 48 down_at 43 last_clean_interval [35,41) [v2:127.0.0.1:6802/112363510,v1:127.0.0.1:6803/112363510] [v2:127.0.0.1:6804/112363510,v1:127.0.0.1:6805/112363510] exists,up 4c996602-e7fe-497f-b8c0-5a182a52c84b 2026-02-22T23:32:22.423 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:32:22.423 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:362: TEST_divergent_ec: wait_for_osd up 1 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:22.424 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:32:22.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:32:22.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:22.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:32:22.626 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 54 up_thru 54 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/1458376431,v1:127.0.0.1:6819/1458376431] [v2:127.0.0.1:6820/1458376431,v1:127.0.0.1:6821/1458376431] exists,up b2449376-0236-402d-9cb6-140a74a80dee 2026-02-22T23:32:22.627 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:32:22.627 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:32:22.627 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:32:22.627 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:363: TEST_divergent_ec: wait_for_osd up 2 2026-02-22T23:32:22.627 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:32:22.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:32:22.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:32:22.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:32:22.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:22.628 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:32:22.628 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:32:22.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:22.629 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:32:22.823 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 48 up_thru 0 down_at 47 last_clean_interval [38,41) [v2:127.0.0.1:6810/3506527952,v1:127.0.0.1:6811/3506527952] [v2:127.0.0.1:6812/3506527952,v1:127.0.0.1:6813/3506527952] exists,up fd920e3e-0aa6-4dbc-9f04-ad94c79a1c32 2026-02-22T23:32:22.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:32:22.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:32:22.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:32:22.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:365: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:23.008 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:23.009 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 51:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:13:05.082410+0000 0 0 2026-02-22T23:32:23.009 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:23.009 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:23.009 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:23.017 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:366: TEST_divergent_ec: echo 'wait for peering' 2026-02-22T23:32:23.017 INFO:tasks.workunit.client.0.trial122.stdout:wait for peering 2026-02-22T23:32:23.018 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:367: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:23.222 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:23.222 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 51:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:13:05.082410+0000 0 0 2026-02-22T23:32:23.222 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:23.222 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:23.223 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:23.232 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:368: TEST_divergent_ec: rados -p test put foo /etc/fstab 2026-02-22T23:32:23.254 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:370: TEST_divergent_ec: echo 'killing divergent 1' 2026-02-22T23:32:23.255 INFO:tasks.workunit.client.0.trial122.stdout:killing divergent 1 2026-02-22T23:32:23.255 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:371: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:23.438 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:23.438 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 51:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:13:05.082410+0000 0 0 2026-02-22T23:32:23.439 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:23.439 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:23.439 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:23.447 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:372: TEST_divergent_ec: kill_daemons td/divergent-priors KILL osd.1 2026-02-22T23:32:23.448 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:32:23.448 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:32:23.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:32:23.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:32:23.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:32:23.556 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:32:23.557 INFO:tasks.workunit.client.0.trial122.stdout:reviving divergent 1 2026-02-22T23:32:23.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:374: TEST_divergent_ec: echo 'reviving divergent 1' 2026-02-22T23:32:23.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:375: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:23.747 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:23.747 INFO:tasks.workunit.client.0.trial122.stdout:1.0 101 0 101 0 0 73767 0 0 18 84 18 active+undersized+degraded 2026-02-22T23:31:44.728845+0000 41'102 51:142 [NONE,0,2] 0 [NONE,0,2] 0 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T05:13:05.082410+0000 0 0 2026-02-22T23:32:23.747 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:23.748 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:23.748 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:23.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:376: TEST_divergent_ec: activate_osd td/divergent-priors 1 2026-02-22T23:32:23.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/divergent-priors 2026-02-22T23:32:23.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:32:23.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:32:23.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:32:23.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/divergent-priors/1 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 ' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/divergent-priors/1' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/divergent-priors/1/journal' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/divergent-priors' 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:32:23.760 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/divergent-priors/$name.log' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/divergent-priors/$name.pid' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:32:23.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:32:23.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:32:23.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:32:23.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/divergent-priors/1 2026-02-22T23:32:23.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:32:23.762 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:32:23.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=e462d5e6-2932-48de-9d0e-5c8a75d6733d --auth-supported=none --mon-host=127.0.0.1:7115 --osd_force_auth_primary_missing_objects=1000000 --osd_debug_pg_log_writeout=true --osd_min_pg_log_entries=12 --osd_max_pg_log_entries=12 --osd_pg_log_trim_min=6 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/divergent-priors/1 --osd-journal=td/divergent-priors/1/journal --chdir= --run-dir=td/divergent-priors '--admin-socket=/tmp/ceph-asok.20359/$cluster-$name.asok' --debug-osd=20 '--log-file=td/divergent-priors/$name.log' '--pid-file=td/divergent-priors/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:32:23.763 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/divergent-priors/1/whoami 2026-02-22T23:32:23.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:32:23.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:32:23.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:32:23.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:32:23.778 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:23.773+0000 7fb092b848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:23.779 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:23.777+0000 7fb092b848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:23.780 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:23.777+0000 7fb092b848c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:23.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:32:23.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:32:23.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:32:23.963 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:32:23.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:32:23.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:23.964 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:32:23.964 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:32:23.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:23.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:32:24.164 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 54 up_thru 54 down_at 30 last_clean_interval [9,28) [v2:127.0.0.1:6818/1458376431,v1:127.0.0.1:6819/1458376431] [v2:127.0.0.1:6820/1458376431,v1:127.0.0.1:6821/1458376431] exists,up b2449376-0236-402d-9cb6-140a74a80dee 2026-02-22T23:32:24.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:32:24.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:32:24.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:32:24.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:378: TEST_divergent_ec: sleep 20 2026-02-22T23:32:25.056 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:25.053+0000 7fb092b848c0 -1 Falling back to public interface 2026-02-22T23:32:28.047 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:28.045+0000 7fb092b848c0 -1 osd.1 55 log_to_monitors true 2026-02-22T23:32:44.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:380: TEST_divergent_ec: echo 'allowing recovery' 2026-02-22T23:32:44.167 INFO:tasks.workunit.client.0.trial122.stdout:allowing recovery 2026-02-22T23:32:44.167 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:381: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:44.354 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:44.354 INFO:tasks.workunit.client.0.trial122.stdout:1.0 102 0 0 0 0 74495 0 0 13 90 13 active+clean 2026-02-22T23:32:28.426675+0000 55'103 59:158 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:08:16.601991+0000 0 0 2026-02-22T23:32:44.354 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:44.355 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:44.355 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:44.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-22T23:32:44.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.0 debug kick_recovery_wq 0 2026-02-22T23:32:44.447 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:32:44.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-22T23:32:44.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.1 debug kick_recovery_wq 0 2026-02-22T23:32:44.532 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:32:44.539 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:383: TEST_divergent_ec: for i in $osds 2026-02-22T23:32:44.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:385: TEST_divergent_ec: ceph tell osd.2 debug kick_recovery_wq 0 2026-02-22T23:32:44.620 INFO:tasks.workunit.client.0.trial122.stderr:kicking recovery queue. set osd_recovery_delay_start to 0 2026-02-22T23:32:44.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:388: TEST_divergent_ec: echo 'reading divergent objects' 2026-02-22T23:32:44.630 INFO:tasks.workunit.client.0.trial122.stdout:reading divergent objects 2026-02-22T23:32:44.630 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:389: TEST_divergent_ec: ceph pg dump pgs 2026-02-22T23:32:44.823 INFO:tasks.workunit.client.0.trial122.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-22T23:32:44.823 INFO:tasks.workunit.client.0.trial122.stdout:1.0 102 0 0 0 0 74495 0 0 13 90 13 active+clean 2026-02-22T23:32:28.426675+0000 55'103 59:158 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-22T23:30:24.904501+0000 0'0 2026-02-22T23:30:24.904501+0000 0 0 periodic scrub scheduled @ 2026-02-24T04:08:16.601991+0000 0 0 2026-02-22T23:32:44.823 INFO:tasks.workunit.client.0.trial122.stdout: 2026-02-22T23:32:44.823 INFO:tasks.workunit.client.0.trial122.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-22T23:32:44.824 INFO:tasks.workunit.client.0.trial122.stderr:dumped pgs 2026-02-22T23:32:44.833 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: expr 2 + 2 2026-02-22T23:32:44.834 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: seq 1 4 2026-02-22T23:32:44.835 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:32:44.835 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_1 td/divergent-priors/existing 2026-02-22T23:32:44.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:32:44.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_2 td/divergent-priors/existing 2026-02-22T23:32:44.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:32:44.885 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_3 td/divergent-priors/existing 2026-02-22T23:32:44.912 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:390: TEST_divergent_ec: for i in $(seq 1 $(expr $DIVERGENT_WRITE + $DIVERGENT_REMOVE)) 2026-02-22T23:32:44.912 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:392: TEST_divergent_ec: rados -p test get existing_4 td/divergent-priors/existing 2026-02-22T23:32:44.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:394: TEST_divergent_ec: rm -f td/divergent-priors/existing 2026-02-22T23:32:44.941 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-22T23:32:44.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:396: TEST_divergent_ec: grep _merge_object_divergent_entries td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:1 entries: 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-22T23:30:36.550665+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: keeping 28'103 (27'99) clone 1:7a80c0d5:::existing_99:1 by unknown.0.0:0 2026-02-22T23:30:36.550665+0000 0 snaps [1] ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version: 27'99 first_divergent_update: 28'103 last_divergent_update: 28'103 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 has no more recent entries in log 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:1 prior_version or op type indicates creation, deleting 2026-02-22T23:32:44.946 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: merging hoid 1:7a80c0d5:::existing_99:head entries: 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4577.0:1 2026-02-22T23:30:41.262677+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: keeping 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4577.0:1 2026-02-22T23:30:41.262677+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head prior_version: 27'99 first_divergent_update: 28'104 last_divergent_update: 28'104 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head has no more recent entries in log 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head olog_can_rollback_to: 28'104 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot rollback 28'104 (27'99) modify 1:7a80c0d5:::existing_99:head by client.4577.0:1 2026-02-22T23:30:41.262677+0000 0 ObjectCleanRegions clean_offsets: [(728, 18446744073709550887)], clean_omap: true, new_object: false 2026-02-22T23:32:44.947 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:7a80c0d5:::existing_99:head cannot roll back, removing and adding to missing 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: merging hoid 1:c156033e:::existing_100:head entries: 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4568.0:1 2026-02-22T23:30:40.193861+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false,27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4573.0:1 2026-02-22T23:30:41.189440+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: keeping 27'101 (27'100) modify 1:c156033e:::existing_100:head by client.4568.0:1 2026-02-22T23:30:40.193861+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 20 _merge_object_divergent_entries: keeping 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4573.0:1 2026-02-22T23:30:41.189440+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid object_not_in_store: 0 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head prior_version: 27'100 first_divergent_update: 27'101 last_divergent_update: 27'102 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head has no more recent entries in log 2026-02-22T23:32:44.948 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head must be rolled back or recovered, attempting to rollback 2026-02-22T23:32:44.949 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head olog_can_rollback_to: 28'104 2026-02-22T23:32:44.949 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot rollback 27'102 (27'101) modify 1:c156033e:::existing_100:head by client.4573.0:1 2026-02-22T23:30:41.189440+0000 0 ObjectCleanRegions clean_offsets: [(967, 18446744073709550648)], clean_omap: true, new_object: false 2026-02-22T23:32:44.949 INFO:tasks.workunit.client.0.trial122.stdout:td/divergent-priors/osd.1.log:2026-02-22T23:32:22.749+0000 7f74f3e62640 10 _merge_object_divergent_entries: hoid 1:c156033e:::existing_100:head cannot roll back, removing and adding to missing 2026-02-22T23:32:44.949 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: find td/divergent-priors -name '*osd*log' 2026-02-22T23:32:44.950 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:405: TEST_divergent_ec: grep -q '_merge_object_divergent_entries.*rolled back' td/divergent-priors/osd.1.log td/divergent-priors/osd.0.log td/divergent-priors/osd.2.log 2026-02-22T23:32:44.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:410: TEST_divergent_ec: echo success 2026-02-22T23:32:44.953 INFO:tasks.workunit.client.0.trial122.stdout:success 2026-02-22T23:32:44.953 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:412: TEST_divergent_ec: delete_pool test 2026-02-22T23:32:44.954 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: delete_pool: local poolname=test 2026-02-22T23:32:44.954 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: delete_pool: ceph osd pool delete test test --yes-i-really-really-mean-it 2026-02-22T23:32:45.203 INFO:tasks.workunit.client.0.trial122.stderr:pool 'test' does not exist 2026-02-22T23:32:45.212 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:413: TEST_divergent_ec: kill_daemons td/divergent-priors 2026-02-22T23:32:45.213 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:32:45.213 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:32:45.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:32:45.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:32:45.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:32:53.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:32:53.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/divergent-priors.sh:47: run: teardown td/divergent-priors 2026-02-22T23:32:53.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:32:53.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:32:53.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:32:53.533 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:32:53.533 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:32:53.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:32:53.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:32:53.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:32:53.539 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:32:53.540 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:32:53.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:32:53.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:32:53.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:32:53.543 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:32:53.543 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:32:53.544 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:32:53.545 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:32:53.545 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:32:53.545 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:32:53.546 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:32:53.547 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:32:53.547 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:32:53.571 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:32:53.571 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.571 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:53.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:32:53.573 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/divergent-priors 0 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/divergent-priors 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/divergent-priors KILL 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:32:53.574 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:32:53.575 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:32:53.576 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:32:53.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:32:53.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:32:53.578 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:32:53.578 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:32:53.579 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:32:53.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:32:53.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:32:53.580 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:32:53.581 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:32:53.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:32:53.583 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-22T23:32:53.583 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/divergent-priors 2026-02-22T23:32:53.585 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:32:53.585 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.585 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.20359 2026-02-22T23:32:53.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.20359 2026-02-22T23:32:53.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:32:53.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:32:53.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-22T23:32:53.588 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-22T23:32:53.589 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-22T23:32:53.639 INFO:tasks.workunit:Running workunit osd/ec-error-rollforward.sh... 2026-02-22T23:32:53.639 DEBUG:teuthology.orchestra.run.trial122:workunit test osd/ec-error-rollforward.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh 2026-02-22T23:32:53.691 INFO:tasks.workunit.client.0.trial122.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-22T23:32:53.696 INFO:tasks.workunit.client.0.trial122.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-22T23:32:53.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:32:53.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:32:53.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-22T23:32:53.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-22T23:32:53.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/ec-error-rollforward 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:6: run: local dir=td/ec-error-rollforward 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:7: run: shift 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: export CEPH_MON=127.0.0.1:7132 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:10: run: CEPH_MON=127.0.0.1:7132 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:11: run: export CEPH_ARGS 2026-02-22T23:32:53.697 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: uuidgen 2026-02-22T23:32:53.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:12: run: CEPH_ARGS+='--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none ' 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:13: run: CEPH_ARGS+='--mon-host=127.0.0.1:7132 ' 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: export margin=10 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:14: run: margin=10 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: export objects=200 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:15: run: objects=200 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: export poolname=test 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:16: run: poolname=test 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: set 2026-02-22T23:32:53.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:18: run: local funcs=TEST_ec_error_rollforward 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:19: run: for func in $funcs 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:20: run: setup td/ec-error-rollforward 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/ec-error-rollforward 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/ec-error-rollforward 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-22T23:32:53.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:32:53.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:32:53.703 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:32:53.703 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:32:53.703 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:32:53.704 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:32:53.704 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:32:53.704 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:32:53.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:32:53.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:32:53.706 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:32:53.706 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:32:53.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:32:53.708 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:32:53.708 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-22T23:32:53.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:32:53.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.710 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:53.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.79600 2026-02-22T23:32:53.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:32:53.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:32:53.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/ec-error-rollforward 2026-02-22T23:32:53.712 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:32:53.713 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.713 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:53.713 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.79600 2026-02-22T23:32:53.714 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:32:53.715 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/ec-error-rollforward 1' TERM HUP INT 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:21: run: TEST_ec_error_rollforward td/ec-error-rollforward 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:27: TEST_ec_error_rollforward: local dir=td/ec-error-rollforward 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:28: TEST_ec_error_rollforward: run_mon td/ec-error-rollforward a 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/ec-error-rollforward 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:32:53.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/ec-error-rollforward/a 2026-02-22T23:32:53.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/ec-error-rollforward/a --run-dir=td/ec-error-rollforward 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:53.739 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:32:53.740 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/ec-error-rollforward/a '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --mon-cluster-log-file=td/ec-error-rollforward/log --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:32:53.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:32:53.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:32:53.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:32:53.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:32:53.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:32:53.770 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:32:53.771 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:32:53.771 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:32:53.771 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:32:53.772 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:32:53.772 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.772 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:53.772 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.79600/ceph-mon.a.asok 2026-02-22T23:32:53.772 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:32:53.772 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.79600/ceph-mon.a.asok config get fsid 2026-02-22T23:32:53.826 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:32:53.826 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:32:53.826 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:32:53.826 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:32:53.826 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:32:53.827 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:32:53.827 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:32:53.827 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:32:53.828 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:32:53.828 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.828 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:53.828 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.79600/ceph-mon.a.asok 2026-02-22T23:32:53.828 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:32:53.828 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.79600/ceph-mon.a.asok config get mon_host 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:29: TEST_ec_error_rollforward: run_mgr td/ec-error-rollforward x 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/ec-error-rollforward 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/ec-error-rollforward/x 2026-02-22T23:32:53.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:53.987 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:32:53.988 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:32:53.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/ec-error-rollforward/x '--log-file=td/ec-error-rollforward/$name.log' '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --run-dir=td/ec-error-rollforward '--pid-file=td/ec-error-rollforward/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:30: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 0 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/0 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:32:54.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:32:54.008 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:32:54.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/0 2026-02-22T23:32:54.010 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:32:54.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=906d66c9-b2af-4241-b08e-7585184cb36d 2026-02-22T23:32:54.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 906d66c9-b2af-4241-b08e-7585184cb36d' 2026-02-22T23:32:54.011 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 906d66c9-b2af-4241-b08e-7585184cb36d 2026-02-22T23:32:54.011 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:32:54.019 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAmkptpmy0mARAA9vszTkjfenE1tFNBTdT0cA== 2026-02-22T23:32:54.019 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAmkptpmy0mARAA9vszTkjfenE1tFNBTdT0cA=="}' 2026-02-22T23:32:54.019 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 906d66c9-b2af-4241-b08e-7585184cb36d -i td/ec-error-rollforward/0/new.json 2026-02-22T23:32:54.127 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:32:54.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/0/new.json 2026-02-22T23:32:54.136 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmkptpmy0mARAA9vszTkjfenE1tFNBTdT0cA== --osd-uuid 906d66c9-b2af-4241-b08e-7585184cb36d 2026-02-22T23:32:54.150 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:54.144+0000 7fcad24fd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:54.151 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:54.148+0000 7fcad24fd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:54.153 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:54.148+0000 7fcad24fd8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:54.153 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:54.148+0000 7fcad24fd8c0 -1 bdev(0x564f13ad9c00 td/ec-error-rollforward/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:32:54.153 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:54.148+0000 7fcad24fd8c0 -1 bluestore(td/ec-error-rollforward/0) _read_fsid unparsable uuid 2026-02-22T23:32:56.758 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/0/keyring 2026-02-22T23:32:56.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:32:56.759 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:32:56.760 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:32:56.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:32:57.023 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:32:57.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:32:57.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:32:57.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:32:57.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:32:57.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:32:57.036 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:57.032+0000 7f9460e768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:57.038 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:57.032+0000 7f9460e768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:57.039 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:57.036+0000 7f9460e768c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:32:57.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:32:57.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:32:57.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:32:57.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:32:57.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:32:57.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:57.215 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:32:57.216 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:32:57.216 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:57.216 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:32:57.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:32:58.414 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:32:58.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:58.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:32:58.415 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:32:58.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:58.415 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:32:58.584 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:32:58.580+0000 7f9460e768c0 -1 Falling back to public interface 2026-02-22T23:32:58.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:32:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:32:59.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:32:59.614 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:32:59.614 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:32:59.614 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:32:59.615 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:32:59.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:00.812 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:00.812 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:00.812 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:33:00.812 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:33:00.812 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:00.812 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:01.007 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:01.862 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:01.856+0000 7f9460e768c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:33:02.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:02.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:02.009 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:33:02.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:33:02.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:02.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:02.213 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:03.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:03.214 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:03.215 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:33:03.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:33:03.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:03.216 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:03.413 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3870689060,v1:127.0.0.1:6803/3870689060] [v2:127.0.0.1:6804/3870689060,v1:127.0.0.1:6805/3870689060] exists,up 906d66c9-b2af-4241-b08e-7585184cb36d 2026-02-22T23:33:03.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:31: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 1 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/1 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:03.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:03.418 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:03.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:33:03.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:33:03.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/1 2026-02-22T23:33:03.420 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:33:03.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=3e24b4e6-61d7-439c-b69c-835baed78736 2026-02-22T23:33:03.420 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 3e24b4e6-61d7-439c-b69c-835baed78736' 2026-02-22T23:33:03.421 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 3e24b4e6-61d7-439c-b69c-835baed78736 2026-02-22T23:33:03.421 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:33:03.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQAvkptp0Z2TGRAAjsAvWaYAiz0JNOHjwtvfJg== 2026-02-22T23:33:03.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQAvkptp0Z2TGRAAjsAvWaYAiz0JNOHjwtvfJg=="}' 2026-02-22T23:33:03.429 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 3e24b4e6-61d7-439c-b69c-835baed78736 -i td/ec-error-rollforward/1/new.json 2026-02-22T23:33:03.630 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:33:03.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/1/new.json 2026-02-22T23:33:03.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAvkptp0Z2TGRAAjsAvWaYAiz0JNOHjwtvfJg== --osd-uuid 3e24b4e6-61d7-439c-b69c-835baed78736 2026-02-22T23:33:03.654 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:03.652+0000 7fafeec618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:03.656 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:03.652+0000 7fafeec618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:03.657 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:03.652+0000 7fafeec618c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:03.658 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:03.652+0000 7fafeec618c0 -1 bdev(0x562f9f059c00 td/ec-error-rollforward/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:33:03.658 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:03.652+0000 7fafeec618c0 -1 bluestore(td/ec-error-rollforward/1) _read_fsid unparsable uuid 2026-02-22T23:33:05.928 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/1/keyring 2026-02-22T23:33:05.928 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:33:05.929 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:33:05.929 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:33:05.929 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:33:06.205 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:33:06.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:33:06.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:06.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:33:06.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:33:06.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:33:06.226 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:06.224+0000 7f04bf8f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:06.228 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:06.224+0000 7f04bf8f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:06.229 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:06.224+0000 7f04bf8f28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:06.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:33:06.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:06.402 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:33:06.403 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:06.403 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:06.403 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:06.403 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:06.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:06.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:06.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:33:06.596 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:07.598 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:07.598 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:07.598 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:33:07.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:33:07.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:07.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:33:07.736 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:07.732+0000 7f04bf8f28c0 -1 Falling back to public interface 2026-02-22T23:33:07.798 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:08.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:08.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:08.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:33:08.800 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:33:08.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:08.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:33:08.996 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:09.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:09.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:09.998 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:33:09.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:33:09.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:09.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:33:10.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:10.734 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:10.728+0000 7f04bf8f28c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:33:11.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:11.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:11.195 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:33:11.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:33:11.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:11.195 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:33:11.392 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3735194233,v1:127.0.0.1:6811/3735194233] [v2:127.0.0.1:6812/3735194233,v1:127.0.0.1:6813/3735194233] exists,up 3e24b4e6-61d7-439c-b69c-835baed78736 2026-02-22T23:33:11.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:32: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 2 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/2 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:11.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:11.396 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:33:11.397 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:33:11.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/2 2026-02-22T23:33:11.398 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:33:11.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=8f7338a0-a733-4b48-a527-d79d2688883a 2026-02-22T23:33:11.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 8f7338a0-a733-4b48-a527-d79d2688883a' 2026-02-22T23:33:11.399 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 8f7338a0-a733-4b48-a527-d79d2688883a 2026-02-22T23:33:11.399 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:33:11.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQA3kptpsJpfGBAAZqa1LP983WIkXR6bwNtAgg== 2026-02-22T23:33:11.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQA3kptpsJpfGBAAZqa1LP983WIkXR6bwNtAgg=="}' 2026-02-22T23:33:11.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 8f7338a0-a733-4b48-a527-d79d2688883a -i td/ec-error-rollforward/2/new.json 2026-02-22T23:33:11.615 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:33:11.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/2/new.json 2026-02-22T23:33:11.625 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQA3kptpsJpfGBAAZqa1LP983WIkXR6bwNtAgg== --osd-uuid 8f7338a0-a733-4b48-a527-d79d2688883a 2026-02-22T23:33:11.639 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:11.636+0000 7f12a80678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:11.641 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:11.636+0000 7f12a80678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:11.642 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:11.640+0000 7f12a80678c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:11.643 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:11.640+0000 7f12a80678c0 -1 bdev(0x558ae5695c00 td/ec-error-rollforward/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:33:11.643 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:11.640+0000 7f12a80678c0 -1 bluestore(td/ec-error-rollforward/2) _read_fsid unparsable uuid 2026-02-22T23:33:13.919 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/2/keyring 2026-02-22T23:33:13.919 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:33:13.920 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:33:13.920 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:33:13.921 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:33:14.198 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:33:14.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:33:14.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:14.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:33:14.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:33:14.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:33:14.219 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:14.216+0000 7f3a928578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:14.221 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:14.216+0000 7f3a928578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:14.222 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:14.216+0000 7f3a928578c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:14.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:33:14.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:14.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:33:14.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:14.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:14.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:14.395 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:14.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:14.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:14.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:14.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:15.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:15.588 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:15.589 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:33:15.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:33:15.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:15.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:15.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:15.988 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:15.984+0000 7f3a928578c0 -1 Falling back to public interface 2026-02-22T23:33:16.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:16.791 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:16.791 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:33:16.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:33:16.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:16.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:16.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:17.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:17.989 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:17.990 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:33:17.990 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:33:17.990 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:17.990 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:18.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:18.949 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:18.944+0000 7f3a928578c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:33:19.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:19.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:19.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:33:19.193 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:33:19.193 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:19.194 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:19.393 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:20.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:20.394 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:20.395 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:33:20.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:33:20.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:20.396 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:20.596 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/722736741,v1:127.0.0.1:6819/722736741] [v2:127.0.0.1:6820/722736741,v1:127.0.0.1:6821/722736741] exists,up 8f7338a0-a733-4b48-a527-d79d2688883a 2026-02-22T23:33:20.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:33: TEST_ec_error_rollforward: run_osd td/ec-error-rollforward 3 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/ec-error-rollforward/3 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:20.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:20.601 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:33:20.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/ec-error-rollforward/3 2026-02-22T23:33:20.603 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:33:20.603 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=ec0ad11b-65ba-46d6-bc8f-63d81692e330 2026-02-22T23:33:20.603 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 ec0ad11b-65ba-46d6-bc8f-63d81692e330' 2026-02-22T23:33:20.604 INFO:tasks.workunit.client.0.trial122.stdout:add osd3 ec0ad11b-65ba-46d6-bc8f-63d81692e330 2026-02-22T23:33:20.604 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:33:20.612 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBAkptpUQCMJBAAK9Rv3HE6riV0kDtVDYjbEA== 2026-02-22T23:33:20.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBAkptpUQCMJBAAK9Rv3HE6riV0kDtVDYjbEA=="}' 2026-02-22T23:33:20.613 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new ec0ad11b-65ba-46d6-bc8f-63d81692e330 -i td/ec-error-rollforward/3/new.json 2026-02-22T23:33:20.815 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:33:20.824 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/ec-error-rollforward/3/new.json 2026-02-22T23:33:20.825 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBAkptpUQCMJBAAK9Rv3HE6riV0kDtVDYjbEA== --osd-uuid ec0ad11b-65ba-46d6-bc8f-63d81692e330 2026-02-22T23:33:20.839 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:20.836+0000 7f4da0a828c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:20.841 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:20.836+0000 7f4da0a828c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:20.842 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:20.840+0000 7f4da0a828c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:20.843 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:20.840+0000 7f4da0a828c0 -1 bdev(0x557549dbdc00 td/ec-error-rollforward/3/block) open stat got: (1) Operation not permitted 2026-02-22T23:33:20.843 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:20.840+0000 7f4da0a828c0 -1 bluestore(td/ec-error-rollforward/3) _read_fsid unparsable uuid 2026-02-22T23:33:23.123 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/ec-error-rollforward/3/keyring 2026-02-22T23:33:23.123 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:33:23.124 INFO:tasks.workunit.client.0.trial122.stdout:adding osd3 key to auth repository 2026-02-22T23:33:23.124 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-22T23:33:23.125 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/ec-error-rollforward/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:33:23.389 INFO:tasks.workunit.client.0.trial122.stdout:start osd.3 2026-02-22T23:33:23.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-22T23:33:23.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:23.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:33:23.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:33:23.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:33:23.410 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:23.408+0000 7f50360158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:23.412 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:23.408+0000 7f50360158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:23.413 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:23.408+0000 7f50360158c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:23.585 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:23.586 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:23.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:23.586 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:33:23.783 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:24.784 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:24.784 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:24.785 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:33:24.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:33:24.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:24.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:33:24.911 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:24.908+0000 7f50360158c0 -1 Falling back to public interface 2026-02-22T23:33:24.979 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:25.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:25.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:25.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:33:25.982 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:33:25.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:25.982 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:33:26.180 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:27.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:27.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:27.182 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:33:27.182 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:33:27.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:27.183 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:33:27.375 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:27.902 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:27.896+0000 7f50360158c0 -1 osd.3 0 log_to_monitors true 2026-02-22T23:33:28.377 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:28.377 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:28.377 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:33:28.377 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:33:28.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:28.378 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:33:28.581 INFO:tasks.workunit.client.0.trial122.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2267937436,v1:127.0.0.1:6827/2267937436] [v2:127.0.0.1:6828/2267937436,v1:127.0.0.1:6829/2267937436] exists,up ec0ad11b-65ba-46d6-bc8f-63d81692e330 2026-02-22T23:33:28.581 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:28.581 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:28.581 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:28.582 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:35: TEST_ec_error_rollforward: ceph osd erasure-code-profile set ec-profile m=2 k=2 crush-failure-domain=osd 2026-02-22T23:33:28.797 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:36: TEST_ec_error_rollforward: ceph osd pool create ec 1 1 erasure ec-profile 2026-02-22T23:33:29.187 INFO:tasks.workunit.client.0.trial122.stderr:pool 'ec' already exists 2026-02-22T23:33:29.197 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:38: TEST_ec_error_rollforward: rados -p ec put foo /etc/passwd 2026-02-22T23:33:29.993 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: cat td/ec-error-rollforward/osd.2.pid 2026-02-22T23:33:29.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:40: TEST_ec_error_rollforward: kill -STOP 85233 2026-02-22T23:33:29.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:43: TEST_ec_error_rollforward: pids=87557 2026-02-22T23:33:29.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:44: TEST_ec_error_rollforward: sleep 1 2026-02-22T23:33:29.995 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:42: TEST_ec_error_rollforward: rados -p ec rm foo 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:46: TEST_ec_error_rollforward: pids+=' 87578' 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:45: TEST_ec_error_rollforward: rados -p ec rm a 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:48: TEST_ec_error_rollforward: pids+=' 87579' 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:47: TEST_ec_error_rollforward: rados -p ec rm b 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:50: TEST_ec_error_rollforward: pids+=' 87580' 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:51: TEST_ec_error_rollforward: sleep 1 2026-02-22T23:33:30.997 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:49: TEST_ec_error_rollforward: rados -p ec rm c 2026-02-22T23:33:31.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:54: TEST_ec_error_rollforward: kill_daemons td/ec-error-rollforward KILL osd 2026-02-22T23:33:31.999 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:31.999 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:32.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:32.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:32.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:32.110 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:32.110 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:55: TEST_ec_error_rollforward: kill 87557 87578 87579 87580 2026-02-22T23:33:32.111 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:56: TEST_ec_error_rollforward: wait 2026-02-22T23:33:32.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:58: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 0 2026-02-22T23:33:32.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:32.114 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/0 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/0' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/0/journal' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:33:32.115 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:32.116 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/0 2026-02-22T23:33:32.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:33:32.118 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:33:32.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/0 --osd-journal=td/ec-error-rollforward/0/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:32.119 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/0/whoami 2026-02-22T23:33:32.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:33:32.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:33:32.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:33:32.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:33:32.133 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.128+0000 7f1c3766e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.134 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.132+0000 7f1c3766e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.135 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.132+0000 7f1c3766e8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:33:32.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:32.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:33:32.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:32.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:32.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:32.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:32.312 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:32.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:32.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:32.509 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3870689060,v1:127.0.0.1:6803/3870689060] [v2:127.0.0.1:6804/3870689060,v1:127.0.0.1:6805/3870689060] exists,up 906d66c9-b2af-4241-b08e-7585184cb36d 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:59: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 1 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:33:32.512 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/1 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/1' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/1/journal' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:32.513 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:32.514 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:33:32.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:33:32.515 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/1 2026-02-22T23:33:32.515 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:33:32.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:33:32.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/1 --osd-journal=td/ec-error-rollforward/1/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:32.516 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/1/whoami 2026-02-22T23:33:32.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:33:32.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:33:32.516 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:33:32.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:33:32.533 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.527+0000 7fca0d3548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.535 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.531+0000 7fca0d3548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.536 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.531+0000 7fca0d3548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:33:32.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:32.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:33:32.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:32.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:32.706 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:32.706 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:32.706 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:32.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:32.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:33:32.900 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 9 up_thru 22 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3735194233,v1:127.0.0.1:6811/3735194233] [v2:127.0.0.1:6812/3735194233,v1:127.0.0.1:6813/3735194233] exists,up 3e24b4e6-61d7-439c-b69c-835baed78736 2026-02-22T23:33:32.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:60: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 2 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/2 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:32.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/2' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/2/journal' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:32.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:33:32.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/2 2026-02-22T23:33:32.907 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/2 --osd-journal=td/ec-error-rollforward/2/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/2/whoami 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:33:32.908 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:33:32.925 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.919+0000 7f9c53e7c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.926 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.923+0000 7f9c53e7c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:32.927 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:32.923+0000 7f9c53e7c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:33.098 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:33.099 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:33.100 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:33.100 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:33:33.293 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/722736741,v1:127.0.0.1:6819/722736741] [v2:127.0.0.1:6820/722736741,v1:127.0.0.1:6821/722736741] exists,up 8f7338a0-a733-4b48-a527-d79d2688883a 2026-02-22T23:33:33.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:33.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:33.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:33.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:61: TEST_ec_error_rollforward: activate_osd td/ec-error-rollforward 3 2026-02-22T23:33:33.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/ec-error-rollforward 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/ec-error-rollforward/3 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 ' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/ec-error-rollforward/3' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/ec-error-rollforward/3/journal' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:33.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/ec-error-rollforward' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/ec-error-rollforward/$name.log' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/ec-error-rollforward/$name.pid' 2026-02-22T23:33:33.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/ec-error-rollforward/3 2026-02-22T23:33:33.299 INFO:tasks.workunit.client.0.trial122.stdout:start osd.3 2026-02-22T23:33:33.300 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-22T23:33:33.300 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=d6d2c64c-3539-43ff-ad1a-5cda3430ee07 --auth-supported=none --mon-host=127.0.0.1:7132 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/ec-error-rollforward/3 --osd-journal=td/ec-error-rollforward/3/journal --chdir= --run-dir=td/ec-error-rollforward '--admin-socket=/tmp/ceph-asok.79600/$cluster-$name.asok' --debug-osd=20 '--log-file=td/ec-error-rollforward/$name.log' '--pid-file=td/ec-error-rollforward/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:33.300 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/ec-error-rollforward/3/whoami 2026-02-22T23:33:33.301 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-22T23:33:33.301 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:33:33.301 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:33:33.301 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:33:33.313 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:33.307+0000 7feb605058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:33.314 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:33.311+0000 7feb605058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:33.316 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:33.311+0000 7feb605058c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:33.407 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:33.403+0000 7f1c3766e8c0 -1 Falling back to public interface 2026-02-22T23:33:33.490 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-22T23:33:33.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:33.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-22T23:33:33.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:33.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:33.491 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:33.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:33.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:33.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:33.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:33:33.693 INFO:tasks.workunit.client.0.trial122.stdout:osd.3 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2267937436,v1:127.0.0.1:6827/2267937436] [v2:127.0.0.1:6828/2267937436,v1:127.0.0.1:6829/2267937436] exists,up ec0ad11b-65ba-46d6-bc8f-63d81692e330 2026-02-22T23:33:33.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:63: TEST_ec_error_rollforward: wait_for_clean 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:33:33.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:33:33.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:33:33.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:33:33.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:33:33.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:33:33.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:33:33.792 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:33:33.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:33:33.993 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:33:33.993 INFO:tasks.workunit.client.0.trial122.stderr:2 2026-02-22T23:33:33.993 INFO:tasks.workunit.client.0.trial122.stderr:3' 2026-02-22T23:33:33.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:33:33.994 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:33:33.994 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:33:34.228 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:34.223+0000 7f9c53e7c8c0 -1 Falling back to public interface 2026-02-22T23:33:34.331 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:34.327+0000 7fca0d3548c0 -1 Falling back to public interface 2026-02-22T23:33:34.840 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:34.835+0000 7feb605058c0 -1 Falling back to public interface 2026-02-22T23:33:36.557 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:36.551+0000 7f1c3766e8c0 -1 osd.0 24 log_to_monitors true 2026-02-22T23:33:37.272 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:37.267+0000 7f9c53e7c8c0 -1 osd.2 23 log_to_monitors true 2026-02-22T23:33:37.470 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:37.463+0000 7fca0d3548c0 -1 osd.1 24 log_to_monitors true 2026-02-22T23:33:38.000 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:37.995+0000 7feb605058c0 -1 osd.3 24 log_to_monitors true 2026-02-22T23:33:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=115964116995 2026-02-22T23:33:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 115964116995 2026-02-22T23:33:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995' 2026-02-22T23:33:39.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:33:39.071 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:33:39.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=137438953475 2026-02-22T23:33:39.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 137438953475 2026-02-22T23:33:39.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995 1-137438953475' 2026-02-22T23:33:39.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:33:39.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:33:39.216 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=128849018883 2026-02-22T23:33:39.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 128849018883 2026-02-22T23:33:39.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995 1-137438953475 2-128849018883' 2026-02-22T23:33:39.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:33:39.217 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-22T23:33:39.291 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822659 2026-02-22T23:33:39.292 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822659 2026-02-22T23:33:39.292 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-115964116995 1-137438953475 2-128849018883 3-154618822659' 2026-02-22T23:33:39.292 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:33:39.292 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-115964116995 2026-02-22T23:33:39.292 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:33:39.293 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:33:39.294 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-115964116995 2026-02-22T23:33:39.294 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:33:39.295 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=115964116995 2026-02-22T23:33:39.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 115964116995' 2026-02-22T23:33:39.296 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 115964116995 2026-02-22T23:33:39.296 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:33:39.490 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 115964116995 2026-02-22T23:33:39.490 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:33:40.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:33:40.492 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:33:40.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-22T23:33:40.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:33:40.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-137438953475 2026-02-22T23:33:40.692 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:33:40.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:33:40.694 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-137438953475 2026-02-22T23:33:40.694 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:33:40.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=137438953475 2026-02-22T23:33:40.695 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 137438953475 2026-02-22T23:33:40.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 137438953475' 2026-02-22T23:33:40.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:33:40.901 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 137438953475 -lt 137438953475 2026-02-22T23:33:40.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:33:40.902 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-128849018883 2026-02-22T23:33:40.903 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:33:40.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:33:40.905 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-128849018883 2026-02-22T23:33:40.905 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:33:40.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=128849018883 2026-02-22T23:33:40.906 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 128849018883' 2026-02-22T23:33:40.906 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 128849018883 2026-02-22T23:33:40.906 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:33:41.106 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 128849018883 -lt 128849018883 2026-02-22T23:33:41.106 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:33:41.106 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-154618822659 2026-02-22T23:33:41.107 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:33:41.108 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-22T23:33:41.108 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-154618822659 2026-02-22T23:33:41.109 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:33:41.110 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822659 2026-02-22T23:33:41.110 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 154618822659' 2026-02-22T23:33:41.110 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.3 seq 154618822659 2026-02-22T23:33:41.110 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-22T23:33:41.306 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822659 -lt 154618822659 2026-02-22T23:33:41.307 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:33:41.307 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:33:41.307 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:33:41.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 1 == 0 2026-02-22T23:33:41.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:33:41.557 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:33:41.558 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:33:41.558 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:33:41.558 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:33:41.558 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:33:41.558 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:33:41.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=1 2026-02-22T23:33:41.749 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:33:41.750 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:33:41.750 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:33:42.002 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 1 = 1 2026-02-22T23:33:42.002 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/ec-error-rollforward.sh:22: run: teardown td/ec-error-rollforward 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:42.003 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:42.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:42.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:42.004 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:42.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:42.314 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:33:42.314 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:33:42.315 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:33:42.316 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:33:42.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:33:42.317 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:33:42.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:42.318 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:33:42.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:33:42.319 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:42.320 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:33:42.321 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:33:42.321 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-22T23:33:42.334 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:33:42.335 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.335 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:42.335 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.79600 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/ec-error-rollforward 0 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/ec-error-rollforward 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/ec-error-rollforward KILL 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:42.337 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:42.338 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:42.338 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:42.338 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:42.339 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:42.339 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:33:42.340 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:33:42.340 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:33:42.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:33:42.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:33:42.341 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:33:42.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:42.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:33:42.343 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:33:42.343 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:42.344 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:33:42.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-22T23:33:42.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/ec-error-rollforward 2026-02-22T23:33:42.347 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:33:42.347 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.347 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.79600 2026-02-22T23:33:42.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.79600 2026-02-22T23:33:42.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:33:42.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:33:42.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-22T23:33:42.349 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-22T23:33:42.350 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-22T23:33:42.402 INFO:tasks.workunit:Running workunit osd/osd-bench.sh... 2026-02-22T23:33:42.402 DEBUG:teuthology.orchestra.run.trial122:workunit test osd/osd-bench.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh 2026-02-22T23:33:42.456 INFO:tasks.workunit.client.0.trial122.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-22T23:33:42.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-bench 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:22: run: local dir=td/osd-bench 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:23: run: shift 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: export CEPH_MON=127.0.0.1:7106 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:25: run: CEPH_MON=127.0.0.1:7106 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:26: run: export CEPH_ARGS 2026-02-22T23:33:42.460 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: uuidgen 2026-02-22T23:33:42.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:27: run: CEPH_ARGS+='--fsid=47d0f3a3-0904-441e-acb5-73d08e763be6 --auth-supported=none ' 2026-02-22T23:33:42.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7106 ' 2026-02-22T23:33:42.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:29: run: CEPH_ARGS+='--debug-bluestore 20 ' 2026-02-22T23:33:42.461 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: set 2026-02-22T23:33:42.461 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-22T23:33:42.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:31: run: local funcs=TEST_bench 2026-02-22T23:33:42.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:32: run: for func in $funcs 2026-02-22T23:33:42.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:33: run: setup td/osd-bench 2026-02-22T23:33:42.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bench 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bench 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:42.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:42.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:42.465 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:33:42.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:33:42.466 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:33:42.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:33:42.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:33:42.468 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:33:42.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:42.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:33:42.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:33:42.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:42.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:33:42.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:33:42.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-22T23:33:42.472 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:33:42.472 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.473 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.90186 2026-02-22T23:33:42.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:33:42.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:33:42.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bench 2026-02-22T23:33:42.475 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:33:42.475 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.475 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.90186 2026-02-22T23:33:42.476 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:33:42.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bench 1' TERM HUP INT 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:34: run: TEST_bench td/osd-bench 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:40: TEST_bench: local dir=td/osd-bench 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:42: TEST_bench: run_mon td/osd-bench a 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bench 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bench/a 2026-02-22T23:33:42.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bench/a --run-dir=td/osd-bench 2026-02-22T23:33:42.508 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:33:42.508 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:42.509 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:42.509 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:42.509 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.509 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.509 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.90186/$cluster-$name.asok' 2026-02-22T23:33:42.510 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bench/a '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.90186/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bench/log --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:33:42.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:33:42.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:33:42.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:33:42.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:33:42.541 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:33:42.542 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.90186/ceph-mon.a.asok 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:33:42.543 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.90186/ceph-mon.a.asok config get fsid 2026-02-22T23:33:42.599 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:33:42.599 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:33:42.599 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:33:42.599 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.90186/ceph-mon.a.asok 2026-02-22T23:33:42.601 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:33:42.602 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.90186/ceph-mon.a.asok config get mon_host 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:43: TEST_bench: run_mgr td/osd-bench x 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bench 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bench/x 2026-02-22T23:33:42.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:33:42.765 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:33:42.765 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:42.765 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:42.766 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:42.766 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.766 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.766 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.90186/$cluster-$name.asok' 2026-02-22T23:33:42.766 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:33:42.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bench/x '--log-file=td/osd-bench/$name.log' '--admin-socket=/tmp/ceph-asok.90186/$cluster-$name.asok' --run-dir=td/osd-bench '--pid-file=td/osd-bench/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:33:42.787 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:44: TEST_bench: run_osd td/osd-bench 0 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bench 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bench/0 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=47d0f3a3-0904-441e-acb5-73d08e763be6 --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 ' 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:42.788 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bench/0' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bench/0/journal' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bench' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.90186/$cluster-$name.asok' 2026-02-22T23:33:42.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.90186/$cluster-$name.asok' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bench/$name.log' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bench/$name.pid' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:33:42.790 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bench/0 2026-02-22T23:33:42.791 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:33:42.791 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=629a72a8-117d-48d3-9ee9-da871f8e2100 2026-02-22T23:33:42.791 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 629a72a8-117d-48d3-9ee9-da871f8e2100' 2026-02-22T23:33:42.792 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 629a72a8-117d-48d3-9ee9-da871f8e2100 2026-02-22T23:33:42.792 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:33:42.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBWkptpvJ2uLxAANABiyKgCz+aOnxipJf45fQ== 2026-02-22T23:33:42.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBWkptpvJ2uLxAANABiyKgCz+aOnxipJf45fQ=="}' 2026-02-22T23:33:42.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 629a72a8-117d-48d3-9ee9-da871f8e2100 -i td/osd-bench/0/new.json 2026-02-22T23:33:42.907 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:42.915 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bench/0/new.json 2026-02-22T23:33:42.916 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=47d0f3a3-0904-441e-acb5-73d08e763be6 --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.90186/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBWkptpvJ2uLxAANABiyKgCz+aOnxipJf45fQ== --osd-uuid 629a72a8-117d-48d3-9ee9-da871f8e2100 2026-02-22T23:33:42.931 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:42.927+0000 7f18c48de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:42.932 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:42.927+0000 7f18c48de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:42.933 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:42.927+0000 7f18c48de8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:42.933 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:42.927+0000 7f18c48de8c0 -1 bdev(0x55f26aa07c00 td/osd-bench/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:33:42.934 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:42.927+0000 7f18c48de8c0 -1 bluestore(td/osd-bench/0) _read_fsid unparsable uuid 2026-02-22T23:33:45.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bench/0/keyring 2026-02-22T23:33:45.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:33:45.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:33:45.736 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:33:45.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bench/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:33:46.008 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:33:46.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:33:46.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=47d0f3a3-0904-441e-acb5-73d08e763be6 --auth-supported=none --mon-host=127.0.0.1:7106 --debug-bluestore 20 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bench/0 --osd-journal=td/osd-bench/0/journal --chdir= --run-dir=td/osd-bench '--admin-socket=/tmp/ceph-asok.90186/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bench/$name.log' '--pid-file=td/osd-bench/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:33:46.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:33:46.008 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:33:46.009 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:33:46.030 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:46.027+0000 7f4238c4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:46.031 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:46.031+0000 7f4238c4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:46.033 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:46.031+0000 7f4238c4c8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:46.201 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:33:46.201 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:33:46.201 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:33:46.201 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:33:46.202 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:33:46.202 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:46.202 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:46.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:33:46.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:46.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:46.398 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:47.399 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:47.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:47.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:33:47.400 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:33:47.400 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:47.401 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:47.539 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:47.539+0000 7f4238c4c8c0 -1 Falling back to public interface 2026-02-22T23:33:47.597 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:48.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:48.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:48.600 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:33:48.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:33:48.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:48.601 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:48.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:49.803 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:49.803 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:49.803 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:33:49.803 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:33:49.803 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:49.803 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:50.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:50.778 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:50.779+0000 7f4238c4c8c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:33:51.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:51.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:51.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:33:51.005 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:33:51.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:51.005 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:51.203 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:33:52.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:33:52.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:33:52.205 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:33:52.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:33:52.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:33:52.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:33:52.403 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/301529842,v1:127.0.0.1:6803/301529842] [v2:127.0.0.1:6804/301529842,v1:127.0.0.1:6805/301529842] exists,up 629a72a8-117d-48d3-9ee9-da871f8e2100 2026-02-22T23:33:52.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:33:52.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:33:52.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:33:52.404 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: CEPH_ARGS= 2026-02-22T23:33:52.404 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: ceph-conf --show-config-value osd_bench_small_size_max_iops 2026-02-22T23:33:52.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:47: TEST_bench: local osd_bench_small_size_max_iops=100 2026-02-22T23:33:52.413 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: CEPH_ARGS= 2026-02-22T23:33:52.414 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: ceph-conf --show-config-value osd_bench_large_size_max_throughput 2026-02-22T23:33:52.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:49: TEST_bench: local osd_bench_large_size_max_throughput=104857600 2026-02-22T23:33:52.428 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: CEPH_ARGS= 2026-02-22T23:33:52.428 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: ceph-conf --show-config-value osd_bench_max_block_size 2026-02-22T23:33:52.441 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:51: TEST_bench: local osd_bench_max_block_size=67108864 2026-02-22T23:33:52.441 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: CEPH_ARGS= 2026-02-22T23:33:52.441 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: ceph-conf --show-config-value osd_bench_duration 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:53: TEST_bench: local osd_bench_duration=30 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:58: TEST_bench: expect_failure td/osd-bench osd_bench_max_block_size ceph tell osd.0 bench 1024 67108865 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=osd_bench_max_block_size 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-22T23:33:52.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 1024 67108865 2026-02-22T23:33:52.527 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-22T23:33:52.527 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-22T23:33:52.527 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet osd_bench_max_block_size td/osd-bench/out 2026-02-22T23:33:52.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-22T23:33:52.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:64: TEST_bench: local bsize=1024 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:65: TEST_bench: local max_count=3072000 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:66: TEST_bench: expect_failure td/osd-bench bench_small_size_max_iops ceph tell osd.0 bench 3072001 1024 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=bench_small_size_max_iops 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-22T23:33:52.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 3072001 1024 2026-02-22T23:33:52.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-22T23:33:52.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-22T23:33:52.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet bench_small_size_max_iops td/osd-bench/out 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:72: TEST_bench: local bsize=1048577 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:73: TEST_bench: local max_count=3145728000 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:74: TEST_bench: expect_failure td/osd-bench osd_bench_large_size_max_throughput ceph tell osd.0 bench 3145728001 1048577 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2025: expect_failure: local dir=td/osd-bench 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2026: expect_failure: shift 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2027: expect_failure: local expected=osd_bench_large_size_max_throughput 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2028: expect_failure: shift 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2029: expect_failure: local success 2026-02-22T23:33:52.602 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2031: expect_failure: ceph tell osd.0 bench 3145728001 1048577 2026-02-22T23:33:52.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2034: expect_failure: success=false 2026-02-22T23:33:52.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: false 2026-02-22T23:33:52.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2037: expect_failure: grep --quiet osd_bench_large_size_max_throughput td/osd-bench/out 2026-02-22T23:33:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2041: expect_failure: return 0 2026-02-22T23:33:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:80: TEST_bench: ceph tell osd.0 bench 2026-02-22T23:33:54.399 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:33:54.399 INFO:tasks.workunit.client.0.trial122.stdout: "bytes_written": 1073741824, 2026-02-22T23:33:54.399 INFO:tasks.workunit.client.0.trial122.stdout: "blocksize": 4194304, 2026-02-22T23:33:54.399 INFO:tasks.workunit.client.0.trial122.stdout: "elapsed_sec": 0.76624824800000002, 2026-02-22T23:33:54.400 INFO:tasks.workunit.client.0.trial122.stdout: "bytes_per_sec": 1401297590.9603646, 2026-02-22T23:33:54.400 INFO:tasks.workunit.client.0.trial122.stdout: "iops": 334.09538053521266 2026-02-22T23:33:54.400 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:33:54.413 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:84: TEST_bench: ceph tell osd.0 bench 10 14456 4444 3 2026-02-22T23:33:54.482 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:33:54.482 INFO:tasks.workunit.client.0.trial122.stdout: "bytes_written": 10, 2026-02-22T23:33:54.483 INFO:tasks.workunit.client.0.trial122.stdout: "blocksize": 14456, 2026-02-22T23:33:54.483 INFO:tasks.workunit.client.0.trial122.stdout: "elapsed_sec": 0.00052607800000000003, 2026-02-22T23:33:54.483 INFO:tasks.workunit.client.0.trial122.stdout: "bytes_per_sec": 19008.588080094585, 2026-02-22T23:33:54.483 INFO:tasks.workunit.client.0.trial122.stdout: "iops": 1.3149272329893875 2026-02-22T23:33:54.483 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:33:54.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:90: TEST_bench: ceph tell osd.0 bench 1 14456 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout:{ 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout: "bytes_written": 1, 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout: "blocksize": 14456, 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout: "elapsed_sec": 0.00038152900000000003, 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout: "bytes_per_sec": 2621.0327393199468, 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout: "iops": 0.18131106387105334 2026-02-22T23:33:54.559 INFO:tasks.workunit.client.0.trial122.stdout:} 2026-02-22T23:33:54.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bench.sh:35: run: teardown td/osd-bench 2026-02-22T23:33:54.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-22T23:33:54.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:33:54.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-22T23:33:54.567 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:54.567 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:54.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:54.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:54.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:54.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:54.676 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:33:54.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:33:54.677 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:33:54.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:33:54.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:33:54.678 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:33:54.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:54.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:33:54.680 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:33:54.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:54.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:33:54.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:33:54.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-22T23:33:54.716 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:33:54.716 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.716 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:54.716 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.90186 2026-02-22T23:33:54.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2377: main: code=0 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-bench 0 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bench 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=0 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bench KILL 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:54.718 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:54.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:54.721 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:33:54.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:33:54.722 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:33:54.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:33:54.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:33:54.723 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:33:54.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:54.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:33:54.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:33:54.725 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:54.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:33:54.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-22T23:33:54.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bench 2026-02-22T23:33:54.728 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:33:54.728 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.728 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.90186 2026-02-22T23:33:54.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.90186 2026-02-22T23:33:54.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:33:54.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:33:54.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 0 2026-02-22T23:33:54.731 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-22T23:33:54.731 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-22T23:33:54.781 INFO:tasks.workunit:Running workunit osd/osd-bluefs-volume-ops.sh... 2026-02-22T23:33:54.781 DEBUG:teuthology.orchestra.run.trial122:workunit test osd/osd-bluefs-volume-ops.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh 2026-02-22T23:33:54.830 INFO:tasks.workunit.client.0.trial122.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: export PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2370: main: PATH=.:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/usr/sbin 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: export PYTHONWARNINGS=ignore 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2371: main: PYTHONWARNINGS=ignore 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: export CEPH_CONF=/dev/null 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2372: main: CEPH_CONF=/dev/null 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2373: main: unset CEPH_ARGS 2026-02-22T23:33:54.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2375: main: local code 2026-02-22T23:33:54.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2376: main: run td/osd-bluefs-volume-ops 2026-02-22T23:33:54.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:8: run: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:54.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:9: run: shift 2026-02-22T23:33:54.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: set 2026-02-22T23:33:54.837 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:11: run: local 'funcs=TEST_bluestore 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:TEST_bluestore2 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:TEST_bluestore_expand' 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:33:54.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-22T23:33:54.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:33:54.840 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:33:54.840 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:33:54.840 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:33:54.840 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:33:54.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:33:54.842 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:33:54.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:33:54.843 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:33:54.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:33:54.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:33:54.844 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:33:54.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:54.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:33:54.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:33:54.846 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:33:54.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:33:54.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:33:54.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-22T23:33:54.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:33:54.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.850 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:54.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92291 2026-02-22T23:33:54.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:33:54.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:33:54.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-22T23:33:54.853 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:33:54.853 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.853 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:54.853 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92291 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 1024 -le 1024 ']' 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:136: setup: ulimit -n 4096 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore td/osd-bluefs-volume-ops 2026-02-22T23:33:54.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:20: TEST_bluestore: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: ulimit -n 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:22: TEST_bluestore: local flimit=4096 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:23: TEST_bluestore: '[' 4096 -lt 1536 ']' 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: export CEPH_MON=127.0.0.1:7146 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:26: TEST_bluestore: CEPH_MON=127.0.0.1:7146 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:27: TEST_bluestore: export CEPH_ARGS 2026-02-22T23:33:54.856 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: uuidgen 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:28: TEST_bluestore: CEPH_ARGS+='--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:29: TEST_bluestore: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:30: TEST_bluestore: CEPH_ARGS+='--bluestore_block_size=2147483648 ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:31: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:32: TEST_bluestore: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:33: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_size=536870912 ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:34: TEST_bluestore: CEPH_ARGS+='--bluestore_block_wal_create=true ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:35: TEST_bluestore: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:37: TEST_bluestore: CEPH_ARGS+='--bluestore_allocation_from_file=1 ' 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:39: TEST_bluestore: run_mon td/osd-bluefs-volume-ops a 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:33:54.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:33:54.859 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-22T23:33:54.859 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:54.881 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:33:54.881 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:54.881 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:54.881 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:54.881 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.881 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:54.882 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:33:54.882 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:33:54.910 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:33:54.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:33:54.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:33:54.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:33:54.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:33:54.911 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:33:54.912 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:33:54.912 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:33:54.912 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:33:54.912 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:33:54.912 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.912 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:54.913 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92291/ceph-mon.a.asok 2026-02-22T23:33:54.913 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:33:54.913 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92291/ceph-mon.a.asok config get fsid 2026-02-22T23:33:54.968 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:33:54.968 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:33:54.968 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:33:54.968 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:33:54.968 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:54.969 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92291/ceph-mon.a.asok 2026-02-22T23:33:54.970 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:33:54.970 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92291/ceph-mon.a.asok config get mon_host 2026-02-22T23:33:55.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:40: TEST_bluestore: run_mgr td/osd-bluefs-volume-ops x 2026-02-22T23:33:55.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:55.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:33:55.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:33:55.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:33:55.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-22T23:33:55.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:33:55.135 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:33:55.135 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:55.135 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:55.136 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:55.136 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:55.136 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:55.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:33:55.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:33:55.137 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:41: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:33:55.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:33:55.155 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:33:55.156 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:33:55.158 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:33:55.158 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 921d18c1-4780-4f78-9901-d3e2b76a9773' 2026-02-22T23:33:55.158 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:33:55.158 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:33:55.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBjkptpP1gTChAALXFUbwLqHXysqcD0A4SSFw== 2026-02-22T23:33:55.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBjkptpP1gTChAALXFUbwLqHXysqcD0A4SSFw=="}' 2026-02-22T23:33:55.168 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 921d18c1-4780-4f78-9901-d3e2b76a9773 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-22T23:33:55.280 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:33:55.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-22T23:33:55.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBjkptpP1gTChAALXFUbwLqHXysqcD0A4SSFw== --osd-uuid 921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:33:55.304 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:55.303+0000 7f1df18148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:55.305 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:55.303+0000 7f1df18148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:55.306 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:55.307+0000 7f1df18148c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:33:55.306 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:55.307+0000 7f1df18148c0 -1 bdev(0x559debafdc00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:33:55.306 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:33:55.307+0000 7f1df18148c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-22T23:34:02.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-22T23:34:02.191 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:34:02.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:34:02.192 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:34:02.192 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:34:02.461 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:34:02.461 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:34:02.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:34:02.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:34:02.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:34:02.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:34:02.484 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:02.483+0000 7f3334f808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:02.485 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:02.483+0000 7f3334f808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:02.486 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:02.487+0000 7f3334f808c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:02.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:02.856 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:03.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:03.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:03.858 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:34:03.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:34:03.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:03.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:04.051 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:04.512 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:04.511+0000 7f3334f808c0 -1 Falling back to public interface 2026-02-22T23:34:05.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:05.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:05.053 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:34:05.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:34:05.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:05.053 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:05.243 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:06.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:06.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:06.245 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:34:06.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:34:06.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:06.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:06.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:07.447 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:07.447 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:07.448 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:34:07.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:34:07.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:07.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:07.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:08.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:08.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:08.641 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:34:08.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:34:08.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:08.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:08.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:09.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:09.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:09.843 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:34:09.844 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:34:09.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:09.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:09.850 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:09.851+0000 7f3334f808c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:34:10.039 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:11.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:11.040 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:11.041 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:34:11.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:34:11.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:11.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:34:11.241 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/169754566,v1:127.0.0.1:6803/169754566] [v2:127.0.0.1:6804/169754566,v1:127.0.0.1:6805/169754566] exists,up 921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:34:11.242 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:34:11.242 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:34:11.242 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:34:11.242 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:34:11.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:42: TEST_bluestore: osd_pid0=93769 2026-02-22T23:34:11.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:43: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 1 2026-02-22T23:34:11.245 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=1 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:34:11.246 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:34:11.247 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:34:11.248 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-22T23:34:11.249 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:34:11.249 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:34:11.249 INFO:tasks.workunit.client.0.trial122.stdout:add osd1 80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:34:11.250 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd1 80ff959e-97d0-4215-a22b-1cc88a9d939f' 2026-02-22T23:34:11.250 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:34:11.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQBzkptpWwd6DxAASg1/aWV7jaKnuXnOyW+gQQ== 2026-02-22T23:34:11.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQBzkptpWwd6DxAASg1/aWV7jaKnuXnOyW+gQQ=="}' 2026-02-22T23:34:11.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 80ff959e-97d0-4215-a22b-1cc88a9d939f -i td/osd-bluefs-volume-ops/1/new.json 2026-02-22T23:34:11.462 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:34:11.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/1/new.json 2026-02-22T23:34:11.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 1 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBzkptpWwd6DxAASg1/aWV7jaKnuXnOyW+gQQ== --osd-uuid 80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:34:11.487 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:11.487+0000 7ff1063f78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:11.488 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:11.487+0000 7ff1063f78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:11.489 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:11.487+0000 7ff1063f78c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:11.490 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:11.487+0000 7ff1063f78c0 -1 bdev(0x563a53713c00 td/osd-bluefs-volume-ops/1/block) open stat got: (1) Operation not permitted 2026-02-22T23:34:11.490 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:11.487+0000 7ff1063f78c0 -1 bluestore(td/osd-bluefs-volume-ops/1) _read_fsid unparsable uuid 2026-02-22T23:34:18.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/1/keyring 2026-02-22T23:34:18.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:34:18.076 INFO:tasks.workunit.client.0.trial122.stdout:adding osd1 key to auth repository 2026-02-22T23:34:18.076 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd1 key to auth repository 2026-02-22T23:34:18.076 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:34:18.347 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:34:18.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.1 2026-02-22T23:34:18.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 1 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:34:18.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:34:18.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:34:18.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:34:18.368 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:18.367+0000 7fb4169fb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:18.369 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:18.367+0000 7fb4169fb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:18.370 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:18.371+0000 7fb4169fb8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:18.539 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 1 2026-02-22T23:34:18.539 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:34:18.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:34:18.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:34:18.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:34:18.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:18.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:34:18.540 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:34:18.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:18.541 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:18.736 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:19.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:19.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:19.738 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:34:19.738 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:34:19.738 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:19.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:19.932 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:20.368 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:20.367+0000 7fb4169fb8c0 -1 Falling back to public interface 2026-02-22T23:34:20.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:20.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:20.933 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:34:20.933 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:34:20.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:20.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:21.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:22.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:22.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:22.130 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:34:22.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:34:22.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:22.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:22.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:23.329 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:23.329 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:23.329 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:34:23.329 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:34:23.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:23.330 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:23.527 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:24.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:24.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:24.528 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:34:24.528 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:34:24.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:24.529 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:24.723 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:25.712 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:25.710+0000 7fb4169fb8c0 -1 osd.1 0 log_to_monitors true 2026-02-22T23:34:25.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:25.724 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:25.724 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:34:25.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:34:25.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:25.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:25.921 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:26.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:26.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:26.923 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:34:26.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:34:26.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:26.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:34:27.127 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 10 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2867055670,v1:127.0.0.1:6811/2867055670] [v2:127.0.0.1:6812/2867055670,v1:127.0.0.1:6813/2867055670] exists,up 80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:34:27.128 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:34:27.128 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:34:27.128 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:34:27.128 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-22T23:34:27.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:44: TEST_bluestore: osd_pid1=96095 2026-02-22T23:34:27.129 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:45: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 2 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=2 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:34:27.130 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:34:27.131 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:34:27.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-22T23:34:27.133 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:34:27.134 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:34:27.134 INFO:tasks.workunit.client.0.trial122.stdout:add osd2 de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:34:27.134 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd2 de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0' 2026-02-22T23:34:27.134 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:34:27.146 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCDkptp1qHFCBAAeoV3DfkvYG2cgM8y4XByuw== 2026-02-22T23:34:27.146 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCDkptp1qHFCBAAeoV3DfkvYG2cgM8y4XByuw=="}' 2026-02-22T23:34:27.146 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 -i td/osd-bluefs-volume-ops/2/new.json 2026-02-22T23:34:27.341 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:34:27.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/2/new.json 2026-02-22T23:34:27.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 2 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCDkptp1qHFCBAAeoV3DfkvYG2cgM8y4XByuw== --osd-uuid de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:34:27.374 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:27.374+0000 7ffae77c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:27.375 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:27.374+0000 7ffae77c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:27.376 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:27.374+0000 7ffae77c18c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:27.377 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:27.374+0000 7ffae77c18c0 -1 bdev(0x5636e936fc00 td/osd-bluefs-volume-ops/2/block) open stat got: (1) Operation not permitted 2026-02-22T23:34:27.377 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:27.374+0000 7ffae77c18c0 -1 bluestore(td/osd-bluefs-volume-ops/2) _read_fsid unparsable uuid 2026-02-22T23:34:33.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/2/keyring 2026-02-22T23:34:33.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:34:33.976 INFO:tasks.workunit.client.0.trial122.stdout:adding osd2 key to auth repository 2026-02-22T23:34:33.976 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd2 key to auth repository 2026-02-22T23:34:33.976 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:34:34.240 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:34:34.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.2 2026-02-22T23:34:34.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 2 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:34:34.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:34:34.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:34:34.241 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:34:34.262 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:34.258+0000 7fecd9a8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:34.263 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:34.262+0000 7fecd9a8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:34.264 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:34.262+0000 7fecd9a8a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:34.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 2 2026-02-22T23:34:34.436 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:34:34.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:34:34.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:34:34.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:34:34.437 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:34.437 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:34:34.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:34:34.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:34.438 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:34.632 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:35.633 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:35.633 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:35.634 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:34:35.634 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:34:35.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:35.635 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:35.827 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:36.252 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:36.250+0000 7fecd9a8a8c0 -1 Falling back to public interface 2026-02-22T23:34:36.828 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:36.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:36.829 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:34:36.829 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:34:36.830 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:36.830 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:37.022 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:38.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:38.023 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:38.024 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:34:38.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:34:38.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:38.025 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:38.215 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:39.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:39.217 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:39.217 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:34:39.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:34:39.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:39.218 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:39.406 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:40.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:40.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:40.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:34:40.409 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:34:40.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:40.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:40.597 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:41.598 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:41.599 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:41.599 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:34:41.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:34:41.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:41.600 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:41.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:41.832 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:41.830+0000 7fecd9a8a8c0 -1 osd.2 0 log_to_monitors true 2026-02-22T23:34:42.791 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:42.791 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:42.791 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:34:42.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:34:42.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:42.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:34:42.980 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 15 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1465299669,v1:127.0.0.1:6819/1465299669] [v2:127.0.0.1:6820/1465299669,v1:127.0.0.1:6821/1465299669] exists,up de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:34:42.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:34:42.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:34:42.981 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:34:42.981 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-22T23:34:42.983 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:46: TEST_bluestore: osd_pid2=98511 2026-02-22T23:34:42.983 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:47: TEST_bluestore: run_osd td/osd-bluefs-volume-ops 3 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=3 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-22T23:34:42.984 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:34:42.985 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:34:42.987 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:34:42.988 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd3 842c9b03-09b3-4c8e-b684-1de46ecf480c' 2026-02-22T23:34:42.989 INFO:tasks.workunit.client.0.trial122.stdout:add osd3 842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:34:42.989 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:34:42.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCSkptpHZ2POxAAkUVvHcLW2UnyswUX6D38rQ== 2026-02-22T23:34:42.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCSkptpHZ2POxAAkUVvHcLW2UnyswUX6D38rQ=="}' 2026-02-22T23:34:42.998 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 842c9b03-09b3-4c8e-b684-1de46ecf480c -i td/osd-bluefs-volume-ops/3/new.json 2026-02-22T23:34:43.195 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:34:43.204 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/3/new.json 2026-02-22T23:34:43.205 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 3 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSkptpHZ2POxAAkUVvHcLW2UnyswUX6D38rQ== --osd-uuid 842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:34:43.221 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:43.218+0000 7fdb92b268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:43.222 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:43.222+0000 7fdb92b268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:43.224 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:43.222+0000 7fdb92b268c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:43.224 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:43.222+0000 7fdb92b268c0 -1 bdev(0x562226d91c00 td/osd-bluefs-volume-ops/3/block) open stat got: (1) Operation not permitted 2026-02-22T23:34:43.224 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:43.222+0000 7fdb92b268c0 -1 bluestore(td/osd-bluefs-volume-ops/3) _read_fsid unparsable uuid 2026-02-22T23:34:49.810 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/3/keyring 2026-02-22T23:34:49.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:34:49.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd3 key to auth repository 2026-02-22T23:34:49.812 INFO:tasks.workunit.client.0.trial122.stdout:adding osd3 key to auth repository 2026-02-22T23:34:49.812 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:34:50.085 INFO:tasks.workunit.client.0.trial122.stdout:start osd.3 2026-02-22T23:34:50.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.3 2026-02-22T23:34:50.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 3 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:34:50.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:34:50.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:34:50.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:34:50.100 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:50.098+0000 7f6a9af0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:50.102 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:50.102+0000 7f6a9af0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:50.103 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:50.102+0000 7f6a9af0b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:34:50.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 3 2026-02-22T23:34:50.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:34:50.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-22T23:34:50.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:34:50.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:34:50.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:50.281 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:34:50.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:34:50.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:50.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:50.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:51.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:51.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:51.480 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:34:51.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:34:51.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:51.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:51.674 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:52.079 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:52.078+0000 7f6a9af0b8c0 -1 Falling back to public interface 2026-02-22T23:34:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:34:52.676 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:34:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:52.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:52.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:53.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:53.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:53.879 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:34:53.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:34:53.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:53.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:54.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:55.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:55.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:55.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:34:55.077 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:34:55.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:55.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:55.276 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:56.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:56.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:56.277 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:34:56.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:34:56.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:56.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:56.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:57.399 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:34:57.398+0000 7f6a9af0b8c0 -1 osd.3 0 log_to_monitors true 2026-02-22T23:34:57.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:57.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:57.479 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:34:57.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:34:57.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:57.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:57.676 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:34:58.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:34:58.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:34:58.678 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:34:58.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:34:58.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:34:58.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:34:58.870 INFO:tasks.workunit.client.0.trial122.stdout:osd.3 up in weight 1 up_from 20 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3583966861,v1:127.0.0.1:6827/3583966861] [v2:127.0.0.1:6828/3583966861,v1:127.0.0.1:6829/3583966861] exists,up 842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:34:58.871 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:34:58.871 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:34:58.871 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:34:58.871 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-22T23:34:58.872 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:48: TEST_bluestore: osd_pid3=100782 2026-02-22T23:34:58.872 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:50: TEST_bluestore: sleep 5 2026-02-22T23:35:03.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:52: TEST_bluestore: create_pool foo 16 2026-02-22T23:35:03.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-22T23:35:04.114 INFO:tasks.workunit.client.0.trial122.stderr:pool 'foo' already exists 2026-02-22T23:35:04.124 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:35:05.125 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:55: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-22T23:35:05.145 INFO:tasks.workunit.client.0.trial122.stdout:hints = 1 2026-02-22T23:35:05.145 INFO:tasks.workunit.client.0.trial122.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-22T23:35:05.145 INFO:tasks.workunit.client.0.trial122.stdout:Object prefix: benchmark_data_trial122_102034 2026-02-22T23:35:05.145 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:35:05.145 INFO:tasks.workunit.client.0.trial122.stdout: 0 0 0 0 0 0 - 0 2026-02-22T23:35:06.145 INFO:tasks.workunit.client.0.trial122.stdout: 1 16 9562 9546 37.2861 37.2891 0.00206585 0.00167371 2026-02-22T23:35:07.145 INFO:tasks.workunit.client.0.trial122.stdout: 2 16 19125 19109 37.3194 37.3555 0.00167297 0.00167271 2026-02-22T23:35:08.145 INFO:tasks.workunit.client.0.trial122.stdout: 3 16 28705 28689 37.3526 37.4219 0.00125216 0.00167126 2026-02-22T23:35:09.145 INFO:tasks.workunit.client.0.trial122.stdout: 4 16 38436 38420 37.5167 38.0117 0.0017589 0.00166413 2026-02-22T23:35:10.145 INFO:tasks.workunit.client.0.trial122.stdout: 5 16 47961 47945 37.4542 37.207 0.00140556 0.00166686 2026-02-22T23:35:11.145 INFO:tasks.workunit.client.0.trial122.stdout: 6 16 57655 57639 37.5225 37.8672 0.00169788 0.00166401 2026-02-22T23:35:12.145 INFO:tasks.workunit.client.0.trial122.stdout: 7 16 67014 66998 37.3844 36.5586 0.00189836 0.00167021 2026-02-22T23:35:13.145 INFO:tasks.workunit.client.0.trial122.stdout: 8 16 77325 77309 37.7457 40.2773 0.00146455 0.00165423 2026-02-22T23:35:14.145 INFO:tasks.workunit.client.0.trial122.stdout: 9 16 87323 87307 37.8908 39.0547 0.00134 0.00164791 2026-02-22T23:35:15.145 INFO:tasks.workunit.client.0.trial122.stdout: 10 16 97405 97389 38.0397 39.3828 0.0015932 0.00164144 2026-02-22T23:35:16.145 INFO:tasks.workunit.client.0.trial122.stdout: 11 16 107800 107784 38.2727 40.6055 0.000785821 0.00163138 2026-02-22T23:35:17.146 INFO:tasks.workunit.client.0.trial122.stdout: 12 16 117682 117666 38.2998 38.6016 0.00152931 0.00163033 2026-02-22T23:35:18.146 INFO:tasks.workunit.client.0.trial122.stdout: 13 16 127989 127973 38.4505 40.2617 0.00146002 0.00162393 2026-02-22T23:35:19.146 INFO:tasks.workunit.client.0.trial122.stdout: 14 16 138434 138418 38.6182 40.8008 0.00136695 0.00161689 2026-02-22T23:35:20.146 INFO:tasks.workunit.client.0.trial122.stdout: 15 16 147538 147522 38.4143 35.5625 0.0018502 0.00162546 2026-02-22T23:35:21.146 INFO:tasks.workunit.client.0.trial122.stdout: 16 16 156371 156355 38.1697 34.5039 0.0016455 0.0016359 2026-02-22T23:35:22.146 INFO:tasks.workunit.client.0.trial122.stdout: 17 16 163183 163167 37.4896 26.6094 0.0738931 0.001663 2026-02-22T23:35:23.146 INFO:tasks.workunit.client.0.trial122.stdout: 18 16 163342 163326 35.4413 0.621094 0.225432 0.00174318 2026-02-22T23:35:24.146 INFO:tasks.workunit.client.0.trial122.stdout: 19 16 163486 163470 33.6056 0.5625 0.0746379 0.00184402 2026-02-22T23:35:25.146 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:35:25.148119+0000 min lat: 0.000610125 max lat: 0.426472 avg lat: 0.00195229 2026-02-22T23:35:25.146 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:35:25.146 INFO:tasks.workunit.client.0.trial122.stdout: 20 16 163652 163636 31.9577 0.648438 0.0915739 0.00195229 2026-02-22T23:35:26.146 INFO:tasks.workunit.client.0.trial122.stdout: 21 16 163801 163785 30.4636 0.582031 0.10665 0.00204161 2026-02-22T23:35:27.146 INFO:tasks.workunit.client.0.trial122.stdout: 22 16 163957 163941 29.1066 0.609375 0.127929 0.00213974 2026-02-22T23:35:28.146 INFO:tasks.workunit.client.0.trial122.stdout: 23 16 170648 170632 28.9774 26.1367 0.00148446 0.00215535 2026-02-22T23:35:29.146 INFO:tasks.workunit.client.0.trial122.stdout: 24 16 179612 179596 29.2289 35.0156 0.00135377 0.00213679 2026-02-22T23:35:30.146 INFO:tasks.workunit.client.0.trial122.stdout: 25 16 188893 188877 29.5098 36.2539 0.00168115 0.00211644 2026-02-22T23:35:31.146 INFO:tasks.workunit.client.0.trial122.stdout: 26 16 197377 197361 29.6493 33.1406 0.00210934 0.00210647 2026-02-22T23:35:32.147 INFO:tasks.workunit.client.0.trial122.stdout: 27 16 206146 206130 29.8198 34.2539 0.0016278 0.00209444 2026-02-22T23:35:33.147 INFO:tasks.workunit.client.0.trial122.stdout: 28 16 214625 214609 29.9376 33.1211 0.00124882 0.0020862 2026-02-22T23:35:34.147 INFO:tasks.workunit.client.0.trial122.stdout: 29 16 223547 223531 30.107 34.8516 0.00170173 0.00207445 2026-02-22T23:35:35.147 INFO:tasks.workunit.client.0.trial122.stdout:Total time run: 30.001 2026-02-22T23:35:35.147 INFO:tasks.workunit.client.0.trial122.stdout:Total writes made: 232592 2026-02-22T23:35:35.147 INFO:tasks.workunit.client.0.trial122.stdout:Write size: 4096 2026-02-22T23:35:35.147 INFO:tasks.workunit.client.0.trial122.stdout:Object size: 4096 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Bandwidth (MB/sec): 30.2845 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Bandwidth: 14.1303 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Max bandwidth (MB/sec): 40.8008 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Min bandwidth (MB/sec): 0.5625 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Average IOPS: 7752 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Stddev IOPS: 3617.37 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Max IOPS: 10445 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Min IOPS: 144 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Average Latency(s): 0.00206231 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Latency(s): 0.00747992 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Max latency(s): 0.426472 2026-02-22T23:35:35.148 INFO:tasks.workunit.client.0.trial122.stdout:Min latency(s): 0.000610125 2026-02-22T23:35:35.150 INFO:tasks.workunit.client.0.trial122.stdout:after bench 2026-02-22T23:35:35.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:57: TEST_bluestore: echo 'after bench' 2026-02-22T23:35:35.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 93769 2026-02-22T23:35:35.151 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: sleep 1 2026-02-22T23:35:36.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:60: TEST_bluestore: kill 93769 2026-02-22T23:35:36.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 60: kill: (93769) - No such process 2026-02-22T23:35:36.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:61: TEST_bluestore: ceph osd down 0 2026-02-22T23:35:36.346 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:35:36.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 96095 2026-02-22T23:35:36.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: sleep 1 2026-02-22T23:35:37.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 96095 2026-02-22T23:35:37.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: sleep 1 2026-02-22T23:35:38.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:62: TEST_bluestore: kill 96095 2026-02-22T23:35:38.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 62: kill: (96095) - No such process 2026-02-22T23:35:38.359 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:63: TEST_bluestore: ceph osd down 1 2026-02-22T23:35:38.550 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:35:38.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 98511 2026-02-22T23:35:38.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: sleep 1 2026-02-22T23:35:39.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:64: TEST_bluestore: kill 98511 2026-02-22T23:35:39.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 64: kill: (98511) - No such process 2026-02-22T23:35:39.561 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:65: TEST_bluestore: ceph osd down 2 2026-02-22T23:35:39.751 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:35:39.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 100782 2026-02-22T23:35:39.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: sleep 1 2026-02-22T23:35:40.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:66: TEST_bluestore: kill 100782 2026-02-22T23:35:40.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 66: kill: (100782) - No such process 2026-02-22T23:35:40.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:67: TEST_bluestore: ceph osd down 3 2026-02-22T23:35:41.013 INFO:tasks.workunit.client.0.trial122.stderr:osd.3 is already down. 2026-02-22T23:35:41.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:70: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:35:44.878 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:35:44.914 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:71: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:35:48.782 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:35:48.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:72: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:35:52.686 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:35:52.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:73: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:35:56.350 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:35:56.384 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:75: TEST_bluestore: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-22T23:35:56.386 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:76: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-22T23:35:56.404 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:35:58.652 INFO:tasks.workunit.client.0.trial122.stdout:0 : device size 0x20000000(512 MiB) : using 0xb600000(182 MiB) 2026-02-22T23:35:58.652 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-22T23:35:58.652 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0x100000000(4 GiB) : using 0xaaa9a000(2.7 GiB) 2026-02-22T23:35:58.653 INFO:tasks.workunit.client.0.trial122.stdout:Expanding DB/WAL... 2026-02-22T23:35:58.654 INFO:tasks.workunit.client.0.trial122.stdout:0 : nothing to do, skipped 2026-02-22T23:35:58.654 INFO:tasks.workunit.client.0.trial122.stdout:1 : nothing to do, skipped 2026-02-22T23:35:58.654 INFO:tasks.workunit.client.0.trial122.stdout:2 : Expanding to 0x100000000(4 GiB) 2026-02-22T23:36:00.090 INFO:tasks.workunit.client.0.trial122.stdout:2 : size updated to 0x100000000(4 GiB) 2026-02-22T23:36:03.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:77: TEST_bluestore: truncate td/osd-bluefs-volume-ops/1/block -s 11811160064 2026-02-22T23:36:03.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:78: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 bluefs-bdev-expand 2026-02-22T23:36:03.857 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:0 : device size 0x20000000(512 MiB) : using 0xb600000(182 MiB) 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0x2c0000000(11 GiB) : using 0x26a993000(9.7 GiB) 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:Expanding DB/WAL... 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:0 : nothing to do, skipped 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:1 : nothing to do, skipped 2026-02-22T23:36:06.058 INFO:tasks.workunit.client.0.trial122.stdout:2 : Expanding to 0x2c0000000(11 GiB) 2026-02-22T23:36:06.059 INFO:tasks.workunit.client.0.trial122.stdout:2 : size updated to 0x2c0000000(11 GiB) 2026-02-22T23:36:09.760 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:79: TEST_bluestore: truncate td/osd-bluefs-volume-ops/2/block -s 4295099392 2026-02-22T23:36:09.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:80: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 bluefs-bdev-expand 2026-02-22T23:36:09.779 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:0 : device size 0x20000000(512 MiB) : using 0xb600000(182 MiB) 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0x100020000(4.0 GiB) : using 0xaa9b0000(2.7 GiB) 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:Expanding DB/WAL... 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:0 : nothing to do, skipped 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:1 : nothing to do, skipped 2026-02-22T23:36:12.005 INFO:tasks.workunit.client.0.trial122.stdout:2 : Expanding to 0x100020000(4.0 GiB) 2026-02-22T23:36:12.006 INFO:tasks.workunit.client.0.trial122.stdout:2 : size updated to 0x100020000(4.0 GiB) 2026-02-22T23:36:15.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:81: TEST_bluestore: truncate td/osd-bluefs-volume-ops/3/block -s 4293918720 2026-02-22T23:36:15.685 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:82: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 bluefs-bdev-expand 2026-02-22T23:36:15.703 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:0 : device size 0x20000000(512 MiB) : using 0xb600000(182 MiB) 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0xc00000(12 MiB) 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0xfff00000(4.0 GiB) : using 0xaa7e2000(2.7 GiB) 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:Expanding DB/WAL... 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:0 : nothing to do, skipped 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:1 : nothing to do, skipped 2026-02-22T23:36:17.895 INFO:tasks.workunit.client.0.trial122.stdout:2 : Expanding to 0xfff00000(4.0 GiB) 2026-02-22T23:36:17.896 INFO:tasks.workunit.client.0.trial122.stdout:2 : size updated to 0xfff00000(4.0 GiB) 2026-02-22T23:36:21.624 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:85: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:36:24.233 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:24.270 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:86: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:36:26.889 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:26.926 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:87: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:36:29.541 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:29.579 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:88: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:36:32.193 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:32.231 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:90: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-22T23:36:32.250 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:33.327 INFO:tasks.workunit.client.0.trial122.stdout:0 : device size 0x20000000(512 MiB) : using 0x800000(8 MiB) 2026-02-22T23:36:33.327 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0x3600000(54 MiB) 2026-02-22T23:36:33.327 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0x100000000(4 GiB) : using 0x2aa9a000(683 MiB) 2026-02-22T23:36:34.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:92: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/block.db --command bluefs-bdev-migrate 2026-02-22T23:36:34.331 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:37.233 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-22T23:36:37.238 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:97: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:36:39.353 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:39.391 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:100: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-22T23:36:39.410 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:42.333 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-22T23:36:42.340 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:105: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:36:44.445 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:44.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:108: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block.wal --devs-source td/osd-bluefs-volume-ops/2/block.db --dev-target td/osd-bluefs-volume-ops/2/block --command bluefs-bdev-migrate 2026-02-22T23:36:44.495 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:47.165 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-22T23:36:47.165 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-22T23:36:47.170 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:114: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:36:48.777 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:48.814 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:117: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block.wal --command bluefs-bdev-migrate 2026-02-22T23:36:48.833 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:49.673 INFO:tasks.workunit.client.0.trial122.stderr:Migrate to WAL device isn't supported. 2026-02-22T23:36:49.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:123: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-22T23:36:49.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:126: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:36:52.281 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:52.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:129: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-22T23:36:52.338 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:55.237 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-22T23:36:55.243 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:134: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:36:57.357 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:36:57.395 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:136: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --dev-target td/osd-bluefs-volume-ops/3/block.db --command bluefs-bdev-migrate 2026-02-22T23:36:57.407 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:36:59.542 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:141: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:37:01.661 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:143: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:37:01.700 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:37:01.701 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:37:01.701 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:37:01.701 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:37:01.701 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:37:01.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:37:01.703 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:37:01.703 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:37:01.703 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:37:01.704 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-22T23:37:01.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:37:01.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:37:01.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:37:01.706 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:37:01.726 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:01.723+0000 7f998f8508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:01.728 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:01.723+0000 7f998f8508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:01.729 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:01.723+0000 7f998f8508c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:01.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:37:01.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:37:01.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:37:01.904 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:37:01.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:37:01.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:01.905 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:37:01.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:37:01.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:01.905 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:02.099 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:03.100 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:03.100 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:03.101 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:37:03.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:37:03.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:03.101 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:03.214 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:03.211+0000 7f998f8508c0 -1 Falling back to public interface 2026-02-22T23:37:03.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:04.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:04.297 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:04.298 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:37:04.298 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:37:04.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:04.299 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:04.490 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:05.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:05.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:05.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:37:05.492 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:37:05.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:05.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:05.686 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:06.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:06.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:06.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:37:06.688 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:37:06.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:06.689 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:06.887 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:07.888 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:07.888 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:07.889 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:37:07.889 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:37:07.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:07.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:08.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:08.425 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:08.419+0000 7f998f8508c0 -1 osd.0 24 log_to_monitors true 2026-02-22T23:37:09.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:09.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:09.085 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:37:09.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:37:09.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:09.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:37:09.278 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 33 up_thru 33 down_at 25 last_clean_interval [5,24) [v2:127.0.0.1:6802/379269526,v1:127.0.0.1:6803/379269526] [v2:127.0.0.1:6804/379269526,v1:127.0.0.1:6805/379269526] exists,up 921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:37:09.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:37:09.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:37:09.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:37:09.279 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:37:09.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:144: TEST_bluestore: osd_pid0=112682 2026-02-22T23:37:09.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:145: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-22T23:37:09.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:37:09.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:37:09.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:37:09.283 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:37:09.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:37:09.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:37:09.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:37:09.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:37:09.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-22T23:37:09.285 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:37:09.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:37:09.286 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:37:09.286 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-22T23:37:09.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:37:09.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:37:09.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:37:09.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:37:09.301 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:09.295+0000 7f55d862b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:09.302 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:09.299+0000 7f55d862b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:09.303 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:09.299+0000 7f55d862b8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:09.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:37:09.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:37:09.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:37:09.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:37:09.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:37:09.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:09.478 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:37:09.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:37:09.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:09.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:09.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:10.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:10.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:10.670 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:37:10.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:37:10.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:10.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:10.783 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:10.783+0000 7f55d862b8c0 -1 Falling back to public interface 2026-02-22T23:37:10.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:11.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:11.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:11.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:37:11.875 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:37:11.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:11.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:12.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:13.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:13.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:13.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:37:13.075 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:37:13.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:13.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:13.277 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:14.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:14.278 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:14.279 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:37:14.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:37:14.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:14.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:14.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:15.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:15.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:15.476 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:37:15.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:37:15.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:15.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:15.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:16.019 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:16.019+0000 7f55d862b8c0 -1 osd.1 26 log_to_monitors true 2026-02-22T23:37:16.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:16.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:16.683 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:37:16.683 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:37:16.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:16.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:37:16.886 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 36 up_thru 36 down_at 27 last_clean_interval [10,26) [v2:127.0.0.1:6810/1182688492,v1:127.0.0.1:6811/1182688492] [v2:127.0.0.1:6812/1182688492,v1:127.0.0.1:6813/1182688492] exists,up 80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:37:16.886 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:37:16.887 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:37:16.887 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:37:16.887 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-22T23:37:16.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:146: TEST_bluestore: osd_pid1=113722 2026-02-22T23:37:16.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:147: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-22T23:37:16.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:37:16.890 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:37:16.891 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:37:16.892 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-22T23:37:16.893 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:37:16.894 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:37:16.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:37:16.895 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-22T23:37:16.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:37:16.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:37:16.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:37:16.895 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:37:16.913 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:16.915+0000 7fa25b61a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:16.915 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:16.915+0000 7fa25b61a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:16.916 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:16.915+0000 7fa25b61a8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:17.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:37:17.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:37:17.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:37:17.091 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:37:17.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:37:17.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:17.092 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:37:17.092 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:37:17.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:17.093 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:17.288 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:18.147 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:18.147+0000 7fa25b61a8c0 -1 Falling back to public interface 2026-02-22T23:37:18.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:18.289 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:18.290 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:37:18.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:37:18.290 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:18.291 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:18.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:19.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:19.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:19.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:37:19.494 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:37:19.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:19.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:19.685 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:20.687 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:20.687 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:20.687 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:37:20.687 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:37:20.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:20.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:20.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:21.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:21.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:21.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:37:21.879 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:37:21.879 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:21.880 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:22.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:23.072 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:23.072 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:23.072 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:37:23.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:37:23.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:23.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:23.132 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:23.131+0000 7fa25b61a8c0 -1 osd.2 28 log_to_monitors true 2026-02-22T23:37:23.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:24.270 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:24.270 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:24.270 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:37:24.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:37:24.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:24.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:37:24.468 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 39 up_thru 39 down_at 29 last_clean_interval [15,28) [v2:127.0.0.1:6818/1011974364,v1:127.0.0.1:6819/1011974364] [v2:127.0.0.1:6820/1011974364,v1:127.0.0.1:6821/1011974364] exists,up de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:37:24.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:37:24.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:37:24.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:37:24.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:148: TEST_bluestore: osd_pid2=114762 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:149: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-22T23:37:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:37:24.474 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:37:24.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:37:24.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:37:24.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-22T23:37:24.476 INFO:tasks.workunit.client.0.trial122.stdout:start osd.3 2026-02-22T23:37:24.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-22T23:37:24.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:37:24.477 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-22T23:37:24.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-22T23:37:24.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:37:24.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:37:24.478 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:37:24.489 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:24.491+0000 7f71618048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:24.490 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:24.491+0000 7f71618048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:24.492 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:24.491+0000 7f71618048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:37:24.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-22T23:37:24.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:37:24.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-22T23:37:24.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:37:24.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:37:24.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:24.670 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:37:24.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:37:24.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:24.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:24.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:25.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:25.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:25.865 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:37:25.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:37:25.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:25.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:26.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:26.203 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:26.203+0000 7f71618048c0 -1 Falling back to public interface 2026-02-22T23:37:27.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:27.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:27.065 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:37:27.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:37:27.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:27.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:27.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:28.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:28.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:28.259 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:37:28.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:37:28.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:28.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:28.452 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:29.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:29.454 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:29.454 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:37:29.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:37:29.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:29.455 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:29.652 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:30.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:30.654 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:30.655 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:37:30.655 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:37:30.655 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:30.656 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:30.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:37:31.450 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:37:31.451+0000 7f71618048c0 -1 osd.3 30 log_to_monitors true 2026-02-22T23:37:31.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:37:31.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:37:31.852 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:37:31.853 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:37:31.853 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:37:31.853 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:37:32.044 INFO:tasks.workunit.client.0.trial122.stdout:osd.3 up in weight 1 up_from 42 up_thru 42 down_at 31 last_clean_interval [20,30) [v2:127.0.0.1:6826/2326785428,v1:127.0.0.1:6827/2326785428] [v2:127.0.0.1:6828/2326785428,v1:127.0.0.1:6829/2326785428] exists,up 842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:37:32.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:37:32.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:37:32.045 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:37:32.045 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-22T23:37:32.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:150: TEST_bluestore: osd_pid3=115797 2026-02-22T23:37:32.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:152: TEST_bluestore: wait_for_clean 2026-02-22T23:37:32.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:37:32.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:37:32.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:37:32.047 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:37:32.048 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:37:32.048 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:37:32.048 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:37:32.048 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:37:32.048 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:37:32.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:37:32.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:37:32.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:37:32.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:37:32.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:37:32.144 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:37:32.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:37:32.348 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:37:32.348 INFO:tasks.workunit.client.0.trial122.stderr:2 2026-02-22T23:37:32.348 INFO:tasks.workunit.client.0.trial122.stderr:3' 2026-02-22T23:37:32.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:37:32.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:37:32.348 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:37:32.423 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=141733920775 2026-02-22T23:37:32.423 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 141733920775 2026-02-22T23:37:32.423 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920775' 2026-02-22T23:37:32.423 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:37:32.424 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:37:32.499 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=154618822662 2026-02-22T23:37:32.499 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 154618822662 2026-02-22T23:37:32.499 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920775 1-154618822662' 2026-02-22T23:37:32.499 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:37:32.499 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:37:32.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=167503724548 2026-02-22T23:37:32.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 167503724548 2026-02-22T23:37:32.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920775 1-154618822662 2-167503724548' 2026-02-22T23:37:32.576 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:37:32.577 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-22T23:37:32.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=180388626435 2026-02-22T23:37:32.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 180388626435 2026-02-22T23:37:32.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-141733920775 1-154618822662 2-167503724548 3-180388626435' 2026-02-22T23:37:32.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:37:32.651 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-141733920775 2026-02-22T23:37:32.651 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:37:32.652 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:37:32.653 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-141733920775 2026-02-22T23:37:32.654 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:37:32.655 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=141733920775 2026-02-22T23:37:32.655 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 141733920775' 2026-02-22T23:37:32.655 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 141733920775 2026-02-22T23:37:32.656 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:37:32.860 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920773 -lt 141733920775 2026-02-22T23:37:32.860 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:37:33.861 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:37:33.861 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:37:34.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 141733920775 -lt 141733920775 2026-02-22T23:37:34.052 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:37:34.053 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-154618822662 2026-02-22T23:37:34.053 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:37:34.054 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:37:34.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-154618822662 2026-02-22T23:37:34.055 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:37:34.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=154618822662 2026-02-22T23:37:34.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 154618822662' 2026-02-22T23:37:34.057 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 154618822662 2026-02-22T23:37:34.057 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:37:34.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 154618822662 -lt 154618822662 2026-02-22T23:37:34.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:37:34.257 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-167503724548 2026-02-22T23:37:34.257 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:37:34.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:37:34.259 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-167503724548 2026-02-22T23:37:34.259 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:37:34.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=167503724548 2026-02-22T23:37:34.261 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 167503724548 2026-02-22T23:37:34.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 167503724548' 2026-02-22T23:37:34.261 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:37:34.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 167503724549 -lt 167503724548 2026-02-22T23:37:34.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:37:34.474 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-180388626435 2026-02-22T23:37:34.474 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:37:34.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-22T23:37:34.476 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-180388626435 2026-02-22T23:37:34.476 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:37:34.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=180388626435 2026-02-22T23:37:34.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 180388626435' 2026-02-22T23:37:34.478 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.3 seq 180388626435 2026-02-22T23:37:34.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-22T23:37:34.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-22T23:37:34.679 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:37:34.679 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:37:34.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:37:34.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-22T23:37:34.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:37:34.937 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:37:34.938 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:37:34.938 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:37:34.938 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:37:34.938 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:37:34.938 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:37:35.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-22T23:37:35.142 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:37:35.143 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:37:35.143 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:37:35.408 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-22T23:37:35.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:37:35.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:37:35.409 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:155: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-22T23:37:35.427 INFO:tasks.workunit.client.0.trial122.stdout:hints = 1 2026-02-22T23:37:35.427 INFO:tasks.workunit.client.0.trial122.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-22T23:37:35.427 INFO:tasks.workunit.client.0.trial122.stdout:Object prefix: benchmark_data_trial122_117205 2026-02-22T23:37:35.427 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:37:35.428 INFO:tasks.workunit.client.0.trial122.stdout: 0 0 0 0 0 0 - 0 2026-02-22T23:37:36.427 INFO:tasks.workunit.client.0.trial122.stdout: 1 15 8822 8807 34.3999 34.4023 0.00171014 0.00181384 2026-02-22T23:37:37.427 INFO:tasks.workunit.client.0.trial122.stdout: 2 16 17383 17367 33.9174 33.4375 0.00186985 0.0018403 2026-02-22T23:37:38.427 INFO:tasks.workunit.client.0.trial122.stdout: 3 16 24440 24424 31.7997 27.5664 0.00145521 0.00196352 2026-02-22T23:37:39.427 INFO:tasks.workunit.client.0.trial122.stdout: 4 16 32862 32846 32.0737 32.8984 0.00187502 0.00194683 2026-02-22T23:37:40.427 INFO:tasks.workunit.client.0.trial122.stdout: 5 16 41184 41168 32.1601 32.5078 0.00162153 0.00194154 2026-02-22T23:37:41.427 INFO:tasks.workunit.client.0.trial122.stdout: 6 16 49600 49584 32.2788 32.875 0.00190056 0.00193452 2026-02-22T23:37:42.428 INFO:tasks.workunit.client.0.trial122.stdout: 7 16 57891 57875 32.2938 32.3867 0.00186163 0.00193369 2026-02-22T23:37:43.428 INFO:tasks.workunit.client.0.trial122.stdout: 8 16 64281 64265 31.3769 24.9609 0.0606173 0.0019788 2026-02-22T23:37:44.428 INFO:tasks.workunit.client.0.trial122.stdout: 9 16 67726 67710 29.3857 13.457 0.00218705 0.00212516 2026-02-22T23:37:45.428 INFO:tasks.workunit.client.0.trial122.stdout: 10 16 76042 76026 29.6953 32.4844 0.00178063 0.00210316 2026-02-22T23:37:46.428 INFO:tasks.workunit.client.0.trial122.stdout: 11 16 84455 84439 29.9831 32.8633 0.0023422 0.0020829 2026-02-22T23:37:47.428 INFO:tasks.workunit.client.0.trial122.stdout: 12 16 92841 92825 30.2141 32.7578 0.00159751 0.002067 2026-02-22T23:37:48.428 INFO:tasks.workunit.client.0.trial122.stdout: 13 16 101284 101268 30.4267 32.9805 0.00231473 0.00205258 2026-02-22T23:37:49.428 INFO:tasks.workunit.client.0.trial122.stdout: 14 16 106740 106724 29.7756 21.3125 0.0578279 0.00209228 2026-02-22T23:37:50.428 INFO:tasks.workunit.client.0.trial122.stdout: 15 16 112139 112123 29.1964 21.0898 0.00206102 0.00213914 2026-02-22T23:37:51.428 INFO:tasks.workunit.client.0.trial122.stdout: 16 15 120453 120438 29.4015 32.4805 0.0016385 0.0021242 2026-02-22T23:37:52.428 INFO:tasks.workunit.client.0.trial122.stdout: 17 16 128877 128861 29.6073 32.9023 0.00141197 0.00210946 2026-02-22T23:37:53.429 INFO:tasks.workunit.client.0.trial122.stdout: 18 16 137304 137288 29.7911 32.918 0.00178258 0.00209642 2026-02-22T23:37:54.429 INFO:tasks.workunit.client.0.trial122.stdout: 19 16 145700 145684 29.9491 32.7969 0.00206299 0.00208539 2026-02-22T23:37:55.429 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:37:55.431031+0000 min lat: 0.000659371 max lat: 0.21776 avg lat: 0.00211975 2026-02-22T23:37:55.429 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:37:55.429 INFO:tasks.workunit.client.0.trial122.stdout: 20 16 150353 150337 29.3604 18.1758 0.119949 0.00211975 2026-02-22T23:37:56.429 INFO:tasks.workunit.client.0.trial122.stdout: 21 16 150653 150637 28.0181 1.17188 0.00212057 0.00222916 2026-02-22T23:37:57.429 INFO:tasks.workunit.client.0.trial122.stdout: 22 16 158941 158925 28.216 32.375 0.0017526 0.00221354 2026-02-22T23:37:58.429 INFO:tasks.workunit.client.0.trial122.stdout: 23 16 167229 167213 28.3967 32.375 0.00194441 0.00219946 2026-02-22T23:37:59.429 INFO:tasks.workunit.client.0.trial122.stdout: 24 16 175588 175572 28.5739 32.6523 0.00143226 0.00218582 2026-02-22T23:38:00.429 INFO:tasks.workunit.client.0.trial122.stdout: 25 16 183746 183730 28.7055 31.8672 0.00197487 0.00217583 2026-02-22T23:38:01.429 INFO:tasks.workunit.client.0.trial122.stdout: 26 16 192018 192002 28.8442 32.3125 0.0018513 0.00216535 2026-02-22T23:38:02.429 INFO:tasks.workunit.client.0.trial122.stdout: 27 16 200382 200366 28.9859 32.6719 0.00168118 0.00215477 2026-02-22T23:38:03.429 INFO:tasks.workunit.client.0.trial122.stdout: 28 16 208551 208535 29.0902 31.9102 0.00158939 0.00214704 2026-02-22T23:38:04.429 INFO:tasks.workunit.client.0.trial122.stdout: 29 16 217097 217081 29.2381 33.3828 0.00159534 0.00213617 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Total time run: 30.0013 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Total writes made: 225409 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Write size: 4096 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Object size: 4096 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Bandwidth (MB/sec): 29.3488 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Bandwidth: 7.52375 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Max bandwidth (MB/sec): 34.4023 2026-02-22T23:38:05.430 INFO:tasks.workunit.client.0.trial122.stdout:Min bandwidth (MB/sec): 1.17188 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Average IOPS: 7513 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Stddev IOPS: 1926.08 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Max IOPS: 8807 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Min IOPS: 300 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Average Latency(s): 0.00212814 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Latency(s): 0.00529168 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Max latency(s): 0.308009 2026-02-22T23:38:05.431 INFO:tasks.workunit.client.0.trial122.stdout:Min latency(s): 0.000659371 2026-02-22T23:38:05.433 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 112682 2026-02-22T23:38:05.433 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: sleep 1 2026-02-22T23:38:06.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:158: TEST_bluestore: kill 112682 2026-02-22T23:38:06.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 158: kill: (112682) - No such process 2026-02-22T23:38:06.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:159: TEST_bluestore: ceph osd down 0 2026-02-22T23:38:06.630 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:38:06.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 113722 2026-02-22T23:38:06.639 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: sleep 1 2026-02-22T23:38:07.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:160: TEST_bluestore: kill 113722 2026-02-22T23:38:07.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 160: kill: (113722) - No such process 2026-02-22T23:38:07.641 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:161: TEST_bluestore: ceph osd down 1 2026-02-22T23:38:07.831 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:38:07.840 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 114762 2026-02-22T23:38:07.841 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: sleep 1 2026-02-22T23:38:08.841 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:162: TEST_bluestore: kill 114762 2026-02-22T23:38:08.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 162: kill: (114762) - No such process 2026-02-22T23:38:08.842 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:163: TEST_bluestore: ceph osd down 2 2026-02-22T23:38:09.033 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:38:09.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 115797 2026-02-22T23:38:09.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: sleep 1 2026-02-22T23:38:10.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:164: TEST_bluestore: kill 115797 2026-02-22T23:38:10.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 164: kill: (115797) - No such process 2026-02-22T23:38:10.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:165: TEST_bluestore: ceph osd down 3 2026-02-22T23:38:10.285 INFO:tasks.workunit.client.0.trial122.stderr:osd.3 is already down. 2026-02-22T23:38:10.294 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:168: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:38:13.905 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:38:13.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:170: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal count=512 bs=1M 2026-02-22T23:38:14.259 INFO:tasks.workunit.client.0.trial122.stderr:512+0 records in 2026-02-22T23:38:14.259 INFO:tasks.workunit.client.0.trial122.stderr:512+0 records out 2026-02-22T23:38:14.259 INFO:tasks.workunit.client.0.trial122.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.311701 s, 1.7 GB/s 2026-02-22T23:38:14.259 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:171: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --dev-target td/osd-bluefs-volume-ops/0/wal --command bluefs-bdev-new-wal 2026-02-22T23:38:14.271 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:38:18.153 INFO:tasks.workunit.client.0.trial122.stdout:WAL device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal 2026-02-22T23:38:18.175 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:175: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:38:22.593 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:38:22.645 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:178: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:38:26.522 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:38:26.562 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:180: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db count=1024 bs=1M 2026-02-22T23:38:27.176 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records in 2026-02-22T23:38:27.176 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records out 2026-02-22T23:38:27.177 INFO:tasks.workunit.client.0.trial122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.613438 s, 1.8 GB/s 2026-02-22T23:38:27.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:181: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --dev-target td/osd-bluefs-volume-ops/1/db --command bluefs-bdev-new-db 2026-02-22T23:38:27.194 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:38:31.582 INFO:tasks.workunit.client.0.trial122.stdout:DB device added /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db 2026-02-22T23:38:31.604 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:185: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block --dev-target td/osd-bluefs-volume-ops/1/block.db --command bluefs-bdev-migrate 2026-02-22T23:38:31.619 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:38:35.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:190: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:38:39.930 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:38:39.977 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:193: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:38:43.086 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:38:43.134 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:195: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-db 2026-02-22T23:38:43.147 INFO:tasks.workunit.client.0.trial122.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-22T23:38:43.147 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:38:45.501 INFO:tasks.workunit.client.0.trial122.stdout:DB device added 2026-02-22T23:38:45.524 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:198: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --command bluefs-bdev-new-wal 2026-02-22T23:38:45.539 INFO:tasks.workunit.client.0.trial122.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-22T23:38:45.540 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:38:48.929 INFO:tasks.workunit.client.0.trial122.stdout:WAL device added 2026-02-22T23:38:48.952 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:201: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/block.db --command bluefs-bdev-migrate 2026-02-22T23:38:48.968 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:38:53.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:206: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:38:57.278 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:38:57.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:209: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:39:00.946 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:39:00.993 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:211: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --command bluefs-bdev-new-wal 2026-02-22T23:39:01.011 INFO:tasks.workunit.client.0.trial122.stdout:NOTICE: --dev-target option omitted, will allocate as a file 2026-02-22T23:39:01.011 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:39:04.133 INFO:tasks.workunit.client.0.trial122.stdout:WAL device added 2026-02-22T23:39:04.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:214: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block.db --dev-target td/osd-bluefs-volume-ops/3/block --command bluefs-bdev-migrate 2026-02-22T23:39:04.172 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:39:08.589 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block.db 2026-02-22T23:39:08.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:219: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:39:12.225 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:39:12.279 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:221: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:39:12.280 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:39:12.281 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:39:12.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:39:12.283 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:39:12.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:39:12.301 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:12.296+0000 7f6f61d048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:12.303 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:12.300+0000 7f6f61d048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:12.304 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:12.300+0000 7f6f61d048c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:12.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:12.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:12.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:13.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:13.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:13.677 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:39:13.678 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:39:13.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:13.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:13.871 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:14.283 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:14.280+0000 7f6f61d048c0 -1 Falling back to public interface 2026-02-22T23:39:14.872 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:14.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:14.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:39:14.873 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:39:14.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:14.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:15.068 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:16.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:16.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:16.070 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:39:16.070 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:39:16.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:16.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:16.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:17.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:17.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:17.268 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:39:17.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:39:17.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:17.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:17.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:18.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:18.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:18.470 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:39:18.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:39:18.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:18.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:18.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:19.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:19.672 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:19.672 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:39:19.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:39:19.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:19.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:19.871 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:20.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:20.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:20.873 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:39:20.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:39:20.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:20.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:21.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:22.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:22.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:22.075 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:39:22.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:39:22.075 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:22.076 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:22.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:23.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:23.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:23.273 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:39:23.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:39:23.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:23.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:23.418 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:23.416+0000 7f6f61d048c0 -1 osd.0 43 log_to_monitors true 2026-02-22T23:39:23.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:24.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:24.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-22T23:39:24.473 INFO:tasks.workunit.client.0.trial122.stdout:10 2026-02-22T23:39:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:24.474 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:39:24.674 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 52 up_thru 52 down_at 44 last_clean_interval [33,43) [v2:127.0.0.1:6802/3195681219,v1:127.0.0.1:6803/3195681219] [v2:127.0.0.1:6804/3195681219,v1:127.0.0.1:6805/3195681219] exists,up 921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:39:24.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:39:24.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:39:24.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:39:24.675 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:39:24.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:222: TEST_bluestore: osd_pid0=123357 2026-02-22T23:39:24.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:223: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-22T23:39:24.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:39:24.678 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:39:24.679 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:39:24.680 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:24.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:24.681 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-22T23:39:24.682 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:39:24.683 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:39:24.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:39:24.684 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-22T23:39:24.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:39:24.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:39:24.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:39:24.684 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:39:24.703 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:24.700+0000 7f394de438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:24.705 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:24.700+0000 7f394de438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:24.706 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:24.704+0000 7f394de438c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:24.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:39:24.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:39:24.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:39:24.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:39:24.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:39:24.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:24.875 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:39:24.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:39:24.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:24.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:25.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:26.076 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:26.076 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:26.076 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:39:26.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:39:26.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:26.077 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:26.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:26.447 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:26.444+0000 7f394de438c0 -1 Falling back to public interface 2026-02-22T23:39:27.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:27.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:27.273 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:39:27.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:39:27.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:27.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:27.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:28.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:28.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:28.471 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:39:28.471 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:39:28.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:28.472 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:28.668 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:29.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:29.670 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:29.670 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:39:29.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:39:29.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:29.671 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:29.863 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:30.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:30.864 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:30.865 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:39:30.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:39:30.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:30.865 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:31.058 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:32.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:32.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:32.060 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:39:32.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:39:32.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:32.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:32.260 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:33.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:33.262 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:33.263 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:39:33.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:39:33.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:33.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:33.456 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:34.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:34.457 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:34.458 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:39:34.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:39:34.458 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:34.459 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:34.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:35.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:35.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:35.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:39:35.662 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:39:35.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:35.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:35.721 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:35.716+0000 7f394de438c0 -1 osd.1 45 log_to_monitors true 2026-02-22T23:39:35.873 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:36.874 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:36.874 INFO:tasks.workunit.client.0.trial122.stdout:10 2026-02-22T23:39:36.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:36.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-22T23:39:36.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:36.875 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:39:37.077 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 55 up_thru 55 down_at 46 last_clean_interval [36,45) [v2:127.0.0.1:6810/2672096411,v1:127.0.0.1:6811/2672096411] [v2:127.0.0.1:6812/2672096411,v1:127.0.0.1:6813/2672096411] exists,up 80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:39:37.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:39:37.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:39:37.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:39:37.079 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-22T23:39:37.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:224: TEST_bluestore: osd_pid1=124502 2026-02-22T23:39:37.081 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:225: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-22T23:39:37.082 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:39:37.083 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:37.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:37.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:39:37.084 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-22T23:39:37.085 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:39:37.086 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:39:37.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:39:37.087 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-22T23:39:37.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:39:37.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:39:37.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:39:37.087 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:39:37.106 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:37.103+0000 7f9dcc4fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:37.108 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:37.103+0000 7f9dcc4fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:37.109 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:37.103+0000 7f9dcc4fc8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:37.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:39:37.284 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:39:37.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:37.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:37.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:38.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:38.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:38.483 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:39:38.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:39:38.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:38.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:38.673 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:38.827 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:38.823+0000 7f9dcc4fc8c0 -1 Falling back to public interface 2026-02-22T23:39:39.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:39.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:39.675 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:39:39.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:39:39.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:39.675 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:39.866 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:40.867 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:40.867 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:40.868 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:39:40.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:39:40.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:40.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:41.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:42.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:42.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:42.062 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:39:42.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:39:42.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:42.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:42.263 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:43.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:43.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:43.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:39:43.266 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:39:43.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:43.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:43.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:44.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:44.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:44.466 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:39:44.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:39:44.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:44.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:44.657 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:45.658 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:45.658 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:45.658 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:39:45.658 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:39:45.659 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:45.659 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:45.855 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:46.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:46.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:46.857 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:39:46.857 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:39:46.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:46.858 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:47.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:47.801 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:47.799+0000 7f9dcc4fc8c0 -1 osd.2 47 log_to_monitors true 2026-02-22T23:39:48.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:48.065 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:39:48.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:48.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:39:48.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:48.066 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:39:48.264 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 58 up_thru 58 down_at 48 last_clean_interval [39,47) [v2:127.0.0.1:6818/3128259224,v1:127.0.0.1:6819/3128259224] [v2:127.0.0.1:6820/3128259224,v1:127.0.0.1:6821/3128259224] exists,up de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:39:48.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:39:48.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:39:48.265 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:39:48.265 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-22T23:39:48.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:226: TEST_bluestore: osd_pid2=125647 2026-02-22T23:39:48.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:227: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-22T23:39:48.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:39:48.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:39:48.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-22T23:39:48.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:39:48.269 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:39:48.270 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:39:48.270 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:39:48.270 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:39:48.270 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:39:48.270 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:48.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:39:48.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:39:48.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:39:48.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-22T23:39:48.272 INFO:tasks.workunit.client.0.trial122.stdout:start osd.3 2026-02-22T23:39:48.273 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:39:48.274 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-22T23:39:48.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-22T23:39:48.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:39:48.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:39:48.274 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:39:48.288 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:48.287+0000 7f97ce8a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:48.289 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:48.291+0000 7f97ce8a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:48.290 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:48.291+0000 7f97ce8a28c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:39:48.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-22T23:39:48.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:39:48.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-22T23:39:48.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:39:48.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:39:48.466 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:48.467 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:39:48.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:39:48.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:48.468 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:48.665 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:49.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:49.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:49.667 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:39:49.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:39:49.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:49.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:49.876 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:50.003 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:50.003+0000 7f97ce8a28c0 -1 Falling back to public interface 2026-02-22T23:39:50.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:50.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:50.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:39:50.877 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:39:50.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:50.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:51.071 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:52.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:52.073 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:52.073 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:39:52.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:39:52.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:52.074 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:52.270 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:53.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:53.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:53.271 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:39:53.272 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:39:53.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:53.272 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:53.467 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:54.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:54.469 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:39:54.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:54.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:39:54.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:54.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:54.667 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:55.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:55.669 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:39:55.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:55.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:39:55.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:55.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:55.867 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:56.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:56.868 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:39:56.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:56.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:39:56.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:56.868 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:57.063 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:58.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:58.064 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:58.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:39:58.065 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:39:58.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:58.065 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:58.266 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:39:58.398 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:39:58.399+0000 7f97ce8a28c0 -1 osd.3 49 log_to_monitors true 2026-02-22T23:39:59.267 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:39:59.268 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:39:59.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:39:59.268 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:39:59.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:39:59.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:39:59.468 INFO:tasks.workunit.client.0.trial122.stdout:osd.3 up in weight 1 up_from 61 up_thru 61 down_at 50 last_clean_interval [42,49) [v2:127.0.0.1:6826/2787912424,v1:127.0.0.1:6827/2787912424] [v2:127.0.0.1:6828/2787912424,v1:127.0.0.1:6829/2787912424] exists,up 842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:39:59.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:39:59.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:39:59.469 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:39:59.469 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-22T23:39:59.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:228: TEST_bluestore: osd_pid3=126767 2026-02-22T23:39:59.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:231: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-22T23:39:59.489 INFO:tasks.workunit.client.0.trial122.stdout:hints = 1 2026-02-22T23:39:59.489 INFO:tasks.workunit.client.0.trial122.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-22T23:39:59.489 INFO:tasks.workunit.client.0.trial122.stdout:Object prefix: benchmark_data_trial122_127867 2026-02-22T23:39:59.489 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:39:59.489 INFO:tasks.workunit.client.0.trial122.stdout: 0 0 0 0 0 0 - 0 2026-02-22T23:40:00.489 INFO:tasks.workunit.client.0.trial122.stdout: 1 16 9922 9906 38.6922 38.6953 0.0014414 0.00161269 2026-02-22T23:40:01.489 INFO:tasks.workunit.client.0.trial122.stdout: 2 16 19448 19432 37.95 37.2109 0.00125844 0.00164486 2026-02-22T23:40:02.489 INFO:tasks.workunit.client.0.trial122.stdout: 3 16 29128 29112 37.9031 37.8125 0.0014524 0.00164706 2026-02-22T23:40:03.489 INFO:tasks.workunit.client.0.trial122.stdout: 4 16 38584 38568 37.661 36.9375 0.00180102 0.0016579 2026-02-22T23:40:04.489 INFO:tasks.workunit.client.0.trial122.stdout: 5 16 48169 48153 37.6165 37.4414 0.00122539 0.00165984 2026-02-22T23:40:05.489 INFO:tasks.workunit.client.0.trial122.stdout: 6 16 57611 57595 37.4938 36.8828 0.0015712 0.00166533 2026-02-22T23:40:06.489 INFO:tasks.workunit.client.0.trial122.stdout: 7 16 67122 67106 37.4446 37.1523 0.00127844 0.00166753 2026-02-22T23:40:07.489 INFO:tasks.workunit.client.0.trial122.stdout: 8 16 76418 76402 37.3027 36.3125 0.00210685 0.00167385 2026-02-22T23:40:08.489 INFO:tasks.workunit.client.0.trial122.stdout: 9 16 85624 85608 37.1533 35.9609 0.00110096 0.00168061 2026-02-22T23:40:09.489 INFO:tasks.workunit.client.0.trial122.stdout: 10 16 89540 89524 34.9676 15.2969 0.00167756 0.00178579 2026-02-22T23:40:10.489 INFO:tasks.workunit.client.0.trial122.stdout: 11 16 98589 98573 35.0019 35.3477 0.0017426 0.00178404 2026-02-22T23:40:11.490 INFO:tasks.workunit.client.0.trial122.stdout: 12 16 107607 107591 35.0204 35.2266 0.00197311 0.00178307 2026-02-22T23:40:12.490 INFO:tasks.workunit.client.0.trial122.stdout: 13 16 116510 116494 35.0015 34.7773 0.00323735 0.00178376 2026-02-22T23:40:13.490 INFO:tasks.workunit.client.0.trial122.stdout: 14 16 125686 125670 35.0615 35.8438 0.00156918 0.00178106 2026-02-22T23:40:14.490 INFO:tasks.workunit.client.0.trial122.stdout: 15 16 132466 132450 34.4895 26.4844 0.0517814 0.00180532 2026-02-22T23:40:15.490 INFO:tasks.workunit.client.0.trial122.stdout: 16 16 137523 137507 33.5684 19.7539 0.00306201 0.00186038 2026-02-22T23:40:16.490 INFO:tasks.workunit.client.0.trial122.stdout: 17 16 146828 146812 33.7317 36.3477 0.00235266 0.00185132 2026-02-22T23:40:17.490 INFO:tasks.workunit.client.0.trial122.stdout: 18 16 155970 155954 33.8416 35.7109 0.00151298 0.00184531 2026-02-22T23:40:18.490 INFO:tasks.workunit.client.0.trial122.stdout: 19 16 165001 164985 33.917 35.2773 0.00135625 0.00184122 2026-02-22T23:40:19.490 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:40:19.492627+0000 min lat: 0.000619048 max lat: 0.174747 avg lat: 0.00183596 2026-02-22T23:40:19.491 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:40:19.491 INFO:tasks.workunit.client.0.trial122.stdout: 20 16 174182 174166 34.0142 35.8633 0.00136728 0.00183596 2026-02-22T23:40:20.490 INFO:tasks.workunit.client.0.trial122.stdout: 21 16 180008 179992 33.4781 22.7578 0.194445 0.00184724 2026-02-22T23:40:21.490 INFO:tasks.workunit.client.0.trial122.stdout: 22 16 180121 180105 31.9764 0.441406 0.213554 0.00194737 2026-02-22T23:40:22.490 INFO:tasks.workunit.client.0.trial122.stdout: 23 16 186044 186028 31.592 23.1367 0.00128868 0.00197682 2026-02-22T23:40:23.491 INFO:tasks.workunit.client.0.trial122.stdout: 24 16 195067 195051 31.7441 35.2461 0.00139855 0.00196739 2026-02-22T23:40:24.491 INFO:tasks.workunit.client.0.trial122.stdout: 25 16 204198 204182 31.901 35.668 0.00156883 0.0019577 2026-02-22T23:40:25.491 INFO:tasks.workunit.client.0.trial122.stdout: 26 16 213228 213212 32.0306 35.2734 0.00144521 0.00194978 2026-02-22T23:40:26.491 INFO:tasks.workunit.client.0.trial122.stdout: 27 16 222827 222811 32.2329 37.4961 0.00539499 0.00193753 2026-02-22T23:40:27.491 INFO:tasks.workunit.client.0.trial122.stdout: 28 16 231836 231820 32.3385 35.1914 0.00181015 0.00193121 2026-02-22T23:40:28.491 INFO:tasks.workunit.client.0.trial122.stdout: 29 16 241769 241753 32.5612 38.8008 0.00173682 0.00191798 2026-02-22T23:40:29.491 INFO:tasks.workunit.client.0.trial122.stdout:Total time run: 30.0008 2026-02-22T23:40:29.491 INFO:tasks.workunit.client.0.trial122.stdout:Total writes made: 251531 2026-02-22T23:40:29.491 INFO:tasks.workunit.client.0.trial122.stdout:Write size: 4096 2026-02-22T23:40:29.491 INFO:tasks.workunit.client.0.trial122.stdout:Object size: 4096 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Bandwidth (MB/sec): 32.7505 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Bandwidth: 8.61542 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Max bandwidth (MB/sec): 38.8008 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Min bandwidth (MB/sec): 0.441406 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Average IOPS: 8384 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Stddev IOPS: 2205.55 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Max IOPS: 9933 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Min IOPS: 113 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Average Latency(s): 0.00190692 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Latency(s): 0.0052182 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Max latency(s): 0.40618 2026-02-22T23:40:29.492 INFO:tasks.workunit.client.0.trial122.stdout:Min latency(s): 0.000619048 2026-02-22T23:40:29.494 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 123357 2026-02-22T23:40:29.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: sleep 1 2026-02-22T23:40:30.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:234: TEST_bluestore: kill 123357 2026-02-22T23:40:30.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 234: kill: (123357) - No such process 2026-02-22T23:40:30.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:235: TEST_bluestore: ceph osd down 0 2026-02-22T23:40:30.685 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:40:30.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 124502 2026-02-22T23:40:30.694 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-22T23:40:31.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 124502 2026-02-22T23:40:31.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: sleep 1 2026-02-22T23:40:32.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:236: TEST_bluestore: kill 124502 2026-02-22T23:40:32.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 236: kill: (124502) - No such process 2026-02-22T23:40:32.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:237: TEST_bluestore: ceph osd down 1 2026-02-22T23:40:33.553 INFO:tasks.workunit.client.0.trial122.stderr:osd.1 is already down. 2026-02-22T23:40:33.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 125647 2026-02-22T23:40:33.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: sleep 1 2026-02-22T23:40:34.566 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 125647 2026-02-22T23:40:34.566 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: sleep 1 2026-02-22T23:40:35.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:238: TEST_bluestore: kill 125647 2026-02-22T23:40:35.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 238: kill: (125647) - No such process 2026-02-22T23:40:35.568 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:239: TEST_bluestore: ceph osd down 2 2026-02-22T23:40:35.762 INFO:tasks.workunit.client.0.trial122.stderr:osd.2 is already down. 2026-02-22T23:40:35.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 126767 2026-02-22T23:40:35.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: sleep 1 2026-02-22T23:40:36.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:240: TEST_bluestore: kill 126767 2026-02-22T23:40:36.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 240: kill: (126767) - No such process 2026-02-22T23:40:36.773 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:241: TEST_bluestore: ceph osd down 3 2026-02-22T23:40:37.015 INFO:tasks.workunit.client.0.trial122.stderr:osd.3 is already down. 2026-02-22T23:40:37.024 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:244: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:40:41.877 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:40:41.943 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:246: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/db2 count=1024 bs=1M 2026-02-22T23:40:42.571 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records in 2026-02-22T23:40:42.571 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records out 2026-02-22T23:40:42.571 INFO:tasks.workunit.client.0.trial122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.626316 s, 1.7 GB/s 2026-02-22T23:40:42.571 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:247: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/db2 --command bluefs-bdev-migrate 2026-02-22T23:40:42.589 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:40:48.753 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-22T23:40:48.754 INFO:tasks.workunit.client.0.trial122.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/db2 2026-02-22T23:40:48.780 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:252: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:40:53.645 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:40:53.708 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:256: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/0/wal2 count=512 bs=1M 2026-02-22T23:40:54.016 INFO:tasks.workunit.client.0.trial122.stderr:512+0 records in 2026-02-22T23:40:54.017 INFO:tasks.workunit.client.0.trial122.stderr:512+0 records out 2026-02-22T23:40:54.017 INFO:tasks.workunit.client.0.trial122.stderr:536870912 bytes (537 MB, 512 MiB) copied, 0.306659 s, 1.8 GB/s 2026-02-22T23:40:54.017 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:257: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.wal --dev-target td/osd-bluefs-volume-ops/0/wal2 --command bluefs-bdev-migrate 2026-02-22T23:40:54.034 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:40:59.190 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/0/block.wal 2026-02-22T23:40:59.190 INFO:tasks.workunit.client.0.trial122.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/0/wal2 2026-02-22T23:40:59.221 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:261: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/0/wal 2026-02-22T23:40:59.261 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:263: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:41:04.113 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:04.174 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:266: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:41:09.033 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:09.095 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:268: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/1/db2 count=1024 bs=1M 2026-02-22T23:41:09.711 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records in 2026-02-22T23:41:09.711 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records out 2026-02-22T23:41:09.711 INFO:tasks.workunit.client.0.trial122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.615219 s, 1.7 GB/s 2026-02-22T23:41:09.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:269: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --devs-source td/osd-bluefs-volume-ops/1/block.wal --dev-target td/osd-bluefs-volume-ops/1/db2 --command bluefs-bdev-migrate 2026-02-22T23:41:09.728 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:41:15.141 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-22T23:41:15.142 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/1/block.wal 2026-02-22T23:41:15.142 INFO:tasks.workunit.client.0.trial122.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/1/db2 2026-02-22T23:41:15.172 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:275: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db 2026-02-22T23:41:15.296 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:277: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:41:19.661 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:19.727 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:279: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 --devs-source td/osd-bluefs-volume-ops/1/block.db --dev-target td/osd-bluefs-volume-ops/1/block --command bluefs-bdev-migrate 2026-02-22T23:41:19.744 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:41:23.625 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/1/block.db 2026-02-22T23:41:23.660 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:284: TEST_bluestore: rm -rf td/osd-bluefs-volume-ops/1/db2 2026-02-22T23:41:23.789 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:286: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/1 fsck 2026-02-22T23:41:27.649 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:27.715 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:289: TEST_bluestore: ceph-objectstore-tool --type bluestore --data-path td/osd-bluefs-volume-ops/2 --op fsck --no-mon-config 2026-02-22T23:41:33.045 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:33.112 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:292: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-22T23:41:33.747 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records in 2026-02-22T23:41:33.748 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records out 2026-02-22T23:41:33.748 INFO:tasks.workunit.client.0.trial122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.634098 s, 1.7 GB/s 2026-02-22T23:41:33.748 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:293: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-22T23:41:33.766 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:41:34.605 INFO:tasks.workunit.client.0.trial122.stderr:Unable to migrate Slow volume to new location, please allocate new DB or WAL with --bluefs-bdev-new-db(wal) command 2026-02-22T23:41:34.608 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:299: TEST_bluestore: '[' 1 -eq 0 ']' 2026-02-22T23:41:34.608 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:302: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:41:40.213 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:40.275 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:305: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/2/db2 count=1024 bs=1M 2026-02-22T23:41:41.766 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records in 2026-02-22T23:41:41.766 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records out 2026-02-22T23:41:41.766 INFO:tasks.workunit.client.0.trial122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.33526 s, 804 MB/s 2026-02-22T23:41:41.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:307: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 --devs-source td/osd-bluefs-volume-ops/2/block --devs-source td/osd-bluefs-volume-ops/2/block.db --devs-source td/osd-bluefs-volume-ops/2/block.wal --dev-target td/osd-bluefs-volume-ops/2/db2 --command bluefs-bdev-migrate 2026-02-22T23:41:41.779 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:41:47.433 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/2/block.db 2026-02-22T23:41:47.434 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/2/block.wal 2026-02-22T23:41:47.434 INFO:tasks.workunit.client.0.trial122.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/2/db2 2026-02-22T23:41:47.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:314: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/2 fsck 2026-02-22T23:41:52.365 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:41:52.428 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:317: TEST_bluestore: dd if=/dev/zero of=td/osd-bluefs-volume-ops/3/wal2 count=1024 bs=1M 2026-02-22T23:41:53.044 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records in 2026-02-22T23:41:53.044 INFO:tasks.workunit.client.0.trial122.stderr:1024+0 records out 2026-02-22T23:41:53.044 INFO:tasks.workunit.client.0.trial122.stderr:1073741824 bytes (1.1 GB, 1.0 GiB) copied, 0.615063 s, 1.7 GB/s 2026-02-22T23:41:53.044 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:319: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 --devs-source td/osd-bluefs-volume-ops/3/block --devs-source td/osd-bluefs-volume-ops/3/block.wal --dev-target td/osd-bluefs-volume-ops/3/wal2 --command bluefs-bdev-migrate 2026-02-22T23:41:53.062 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:41:57.697 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/3/block 2026-02-22T23:41:57.698 INFO:tasks.workunit.client.0.trial122.stdout: device removed:0 td/osd-bluefs-volume-ops/3/block.wal 2026-02-22T23:41:57.698 INFO:tasks.workunit.client.0.trial122.stdout: device added: 1 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td/osd-bluefs-volume-ops/3/wal2 2026-02-22T23:41:57.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:325: TEST_bluestore: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/3 fsck 2026-02-22T23:42:02.097 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:327: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:42:02.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:42:02.162 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:42:02.163 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:42:02.164 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:42:02.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:42:02.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:42:02.165 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-22T23:42:02.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:42:02.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:42:02.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:42:02.166 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:42:02.177 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:02.172+0000 7f2df46db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:02.179 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:02.176+0000 7f2df46db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:02.179 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:02.176+0000 7f2df46db8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:02.360 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:42:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:42:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:42:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:42:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:42:02.361 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:02.361 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:42:02.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:42:02.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:02.362 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:02.552 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:03.553 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:03.553 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:03.554 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:42:03.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:42:03.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:03.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:03.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:03.919 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:03.916+0000 7f2df46db8c0 -1 Falling back to public interface 2026-02-22T23:42:04.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:04.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:04.752 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:42:04.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:42:04.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:04.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:04.944 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:05.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:05.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:05.945 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:42:05.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:42:05.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:05.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:06.147 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:07.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:07.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:07.149 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:42:07.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:42:07.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:07.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:07.348 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:08.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:08.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:08.351 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:42:08.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:42:08.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:08.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:08.548 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:09.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:09.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:09.550 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:42:09.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:42:09.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:09.550 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:09.748 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:10.750 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:42:10.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:42:10.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:10.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:10.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:11.947 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:11.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:11.948 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:42:11.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:42:11.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:11.949 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:12.147 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:13.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:13.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:13.149 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:42:13.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:42:13.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:13.150 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:13.349 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:14.008 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:14.004+0000 7f2df46db8c0 -1 osd.0 62 log_to_monitors true 2026-02-22T23:42:14.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:14.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:14.351 INFO:tasks.workunit.client.0.trial122.stdout:10 2026-02-22T23:42:14.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-22T23:42:14.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:14.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:42:14.550 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 71 up_thru 71 down_at 63 last_clean_interval [52,62) [v2:127.0.0.1:6802/2703306478,v1:127.0.0.1:6803/2703306478] [v2:127.0.0.1:6804/2703306478,v1:127.0.0.1:6805/2703306478] exists,up 921d18c1-4780-4f78-9901-d3e2b76a9773 2026-02-22T23:42:14.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:42:14.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:42:14.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:42:14.551 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:328: TEST_bluestore: osd_pid0=134073 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:329: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 1 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=1 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/1 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:42:14.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/1' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/1/journal' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:42:14.555 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:42:14.556 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:42:14.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/1 2026-02-22T23:42:14.558 INFO:tasks.workunit.client.0.trial122.stdout:start osd.1 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.1 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 1 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/1 --osd-journal=td/osd-bluefs-volume-ops/1/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/1/whoami 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 1 = 1 ']' 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:42:14.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:42:14.578 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:14.576+0000 7fa532b038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:14.579 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:14.576+0000 7fa532b038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:14.580 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:14.576+0000 7fa532b038c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:14.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 1 2026-02-22T23:42:14.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:42:14.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=1 2026-02-22T23:42:14.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:42:14.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:42:14.751 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:14.752 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:42:14.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:42:14.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:14.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:14.955 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:15.807 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:15.804+0000 7fa532b038c0 -1 Falling back to public interface 2026-02-22T23:42:15.956 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:15.956 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:15.957 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:42:15.957 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:42:15.957 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:15.958 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:16.159 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:17.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:17.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:17.161 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:42:17.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:42:17.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:17.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:17.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:18.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:18.366 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:18.367 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:42:18.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:42:18.367 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:18.368 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:18.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:19.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:19.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:19.566 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:42:19.566 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:42:19.566 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:19.567 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:19.761 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:20.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:20.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:20.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:42:20.764 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:42:20.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:20.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:20.955 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:21.956 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:21.957 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:21.957 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:42:21.958 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:42:21.958 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:21.958 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:22.152 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:23.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:23.154 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:23.154 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:42:23.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:42:23.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:23.155 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:23.350 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:24.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:24.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:24.352 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:42:24.352 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:42:24.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:24.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:24.555 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:24.766 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:24.764+0000 7fa532b038c0 -1 osd.1 64 log_to_monitors true 2026-02-22T23:42:25.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:25.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:25.557 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:42:25.557 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:42:25.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:25.558 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.1 up' 2026-02-22T23:42:25.762 INFO:tasks.workunit.client.0.trial122.stdout:osd.1 up in weight 1 up_from 74 up_thru 74 down_at 65 last_clean_interval [55,64) [v2:127.0.0.1:6810/131298680,v1:127.0.0.1:6811/131298680] [v2:127.0.0.1:6812/131298680,v1:127.0.0.1:6813/131298680] exists,up 80ff959e-97d0-4215-a22b-1cc88a9d939f 2026-02-22T23:42:25.762 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:42:25.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:42:25.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:42:25.763 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.1.pid 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:330: TEST_bluestore: osd_pid1=135218 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:331: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 2 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=2 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/2 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:42:25.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/2' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/2/journal' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:42:25.768 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:42:25.769 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/2 2026-02-22T23:42:25.770 INFO:tasks.workunit.client.0.trial122.stdout:start osd.2 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.2 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 2 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/2 --osd-journal=td/osd-bluefs-volume-ops/2/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/2/whoami 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 2 = 2 ']' 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:42:25.771 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:42:25.785 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:25.784+0000 7f332862f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:25.787 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:25.784+0000 7f332862f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:25.787 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:25.784+0000 7f332862f8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:25.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 2 2026-02-22T23:42:25.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:42:25.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=2 2026-02-22T23:42:25.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:42:25.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:42:25.967 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:25.968 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:42:25.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:42:25.968 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:25.969 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:26.159 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:27.160 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:27.161 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:27.161 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:42:27.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:42:27.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:27.162 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:27.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:27.503 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:27.500+0000 7f332862f8c0 -1 Falling back to public interface 2026-02-22T23:42:28.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:28.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:28.358 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:42:28.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:42:28.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:28.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:28.551 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:29.552 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:29.553 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:29.553 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:42:29.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:42:29.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:29.554 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:29.744 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:30.745 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:30.746 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:30.746 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:42:30.747 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:42:30.747 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:30.747 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:30.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:31.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:31.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:31.946 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:42:31.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:42:31.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:31.946 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:32.142 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:33.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:33.144 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:33.144 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:42:33.145 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:42:33.145 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:33.145 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:33.339 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:34.340 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:34.341 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:34.341 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:42:34.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:42:34.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:34.342 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:34.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:35.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:35.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:35.536 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:42:35.537 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:42:35.537 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:35.537 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:35.732 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:36.733 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:36.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:36.734 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:42:36.734 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:42:36.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:36.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:36.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:37.848 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:37.843+0000 7f332862f8c0 -1 osd.2 66 log_to_monitors true 2026-02-22T23:42:37.937 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:37.937 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:37.938 INFO:tasks.workunit.client.0.trial122.stdout:10 2026-02-22T23:42:37.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-22T23:42:37.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:37.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.2 up' 2026-02-22T23:42:38.133 INFO:tasks.workunit.client.0.trial122.stdout:osd.2 up in weight 1 up_from 78 up_thru 78 down_at 67 last_clean_interval [58,66) [v2:127.0.0.1:6818/2197309852,v1:127.0.0.1:6819/2197309852] [v2:127.0.0.1:6820/2197309852,v1:127.0.0.1:6821/2197309852] exists,up de76fe45-9f58-43d3-a8fc-3b0ad1d2fce0 2026-02-22T23:42:38.133 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:42:38.133 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:42:38.133 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:42:38.134 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.2.pid 2026-02-22T23:42:38.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:332: TEST_bluestore: osd_pid2=136328 2026-02-22T23:42:38.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:333: TEST_bluestore: activate_osd td/osd-bluefs-volume-ops 3 2026-02-22T23:42:38.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:42:38.138 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=3 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/3 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 ' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/3' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/3/journal' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:42:38.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:42:38.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:42:38.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/3 2026-02-22T23:42:38.142 INFO:tasks.workunit.client.0.trial122.stdout:start osd.3 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.3 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 3 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/3 --osd-journal=td/osd-bluefs-volume-ops/3/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/3/whoami 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 3 = 3 ']' 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:42:38.143 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:42:38.162 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:38.159+0000 7f9d225ee8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:38.163 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:38.159+0000 7f9d225ee8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:38.164 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:38.159+0000 7f9d225ee8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:42:38.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 3 2026-02-22T23:42:38.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:42:38.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=3 2026-02-22T23:42:38.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:42:38.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:42:38.334 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:38.335 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:42:38.335 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:42:38.335 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:38.335 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:38.537 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:39.539 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:39.539 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:39.539 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:42:39.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:42:39.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:39.540 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:39.647 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:39.643+0000 7f9d225ee8c0 -1 Falling back to public interface 2026-02-22T23:42:39.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:40.740 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:40.740 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:40.741 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:42:40.741 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:42:40.742 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:40.742 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:40.938 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:41.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:41.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:41.939 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:42:41.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:42:41.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:41.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:42.139 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:43.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:43.140 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:43.141 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:42:43.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:42:43.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:43.141 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:43.336 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:44.338 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:44.338 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:44.338 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:42:44.339 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:42:44.339 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:44.339 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:44.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:45.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:45.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:45.535 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:42:45.536 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:42:45.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:45.536 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:45.726 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:46.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:46.728 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:46.729 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:42:46.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:42:46.729 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:46.730 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:46.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:47.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:47.925 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:47.925 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:42:47.926 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:42:47.926 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:47.926 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:48.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:42:48.811 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:42:48.811+0000 7f9d225ee8c0 -1 osd.3 68 log_to_monitors true 2026-02-22T23:42:49.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:42:49.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:42:49.120 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:42:49.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:42:49.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:42:49.120 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.3 up' 2026-02-22T23:42:49.321 INFO:tasks.workunit.client.0.trial122.stdout:osd.3 up in weight 1 up_from 81 up_thru 81 down_at 69 last_clean_interval [61,68) [v2:127.0.0.1:6826/3391803983,v1:127.0.0.1:6827/3391803983] [v2:127.0.0.1:6828/3391803983,v1:127.0.0.1:6829/3391803983] exists,up 842c9b03-09b3-4c8e-b684-1de46ecf480c 2026-02-22T23:42:49.321 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:42:49.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:42:49.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:42:49.322 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: cat td/osd-bluefs-volume-ops/osd.3.pid 2026-02-22T23:42:49.322 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:334: TEST_bluestore: osd_pid3=137468 2026-02-22T23:42:49.323 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:337: TEST_bluestore: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-22T23:42:49.340 INFO:tasks.workunit.client.0.trial122.stdout:hints = 1 2026-02-22T23:42:49.340 INFO:tasks.workunit.client.0.trial122.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-22T23:42:49.340 INFO:tasks.workunit.client.0.trial122.stdout:Object prefix: benchmark_data_trial122_138568 2026-02-22T23:42:49.340 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:42:49.340 INFO:tasks.workunit.client.0.trial122.stdout: 0 0 0 0 0 0 - 0 2026-02-22T23:42:50.340 INFO:tasks.workunit.client.0.trial122.stdout: 1 16 10380 10364 40.4813 40.4844 0.00318278 0.00154115 2026-02-22T23:42:51.340 INFO:tasks.workunit.client.0.trial122.stdout: 2 16 21040 21024 41.0595 41.6406 0.00131365 0.00152022 2026-02-22T23:42:52.340 INFO:tasks.workunit.client.0.trial122.stdout: 3 16 31166 31150 40.5568 39.5547 0.0015407 0.00153901 2026-02-22T23:42:53.340 INFO:tasks.workunit.client.0.trial122.stdout: 4 15 41361 41346 40.3739 39.8281 0.00154312 0.00154638 2026-02-22T23:42:54.340 INFO:tasks.workunit.client.0.trial122.stdout: 5 16 51607 51591 40.3024 40.0195 0.00125795 0.00154912 2026-02-22T23:42:55.340 INFO:tasks.workunit.client.0.trial122.stdout: 6 16 61737 61721 40.1799 39.5703 0.00154788 0.0015539 2026-02-22T23:42:56.340 INFO:tasks.workunit.client.0.trial122.stdout: 7 16 71398 71382 39.8306 37.7383 0.00159819 0.00156749 2026-02-22T23:42:57.341 INFO:tasks.workunit.client.0.trial122.stdout: 8 16 80827 80811 39.4555 36.832 0.00173172 0.00158245 2026-02-22T23:42:58.341 INFO:tasks.workunit.client.0.trial122.stdout: 9 16 87750 87734 38.0761 27.043 0.0831327 0.00163243 2026-02-22T23:42:59.341 INFO:tasks.workunit.client.0.trial122.stdout: 10 16 94324 94308 36.8362 25.6797 0.00149064 0.00169506 2026-02-22T23:43:00.341 INFO:tasks.workunit.client.0.trial122.stdout: 11 16 103800 103784 36.8523 37.0156 0.00147296 0.00169434 2026-02-22T23:43:01.341 INFO:tasks.workunit.client.0.trial122.stdout: 12 16 113243 113227 36.8549 36.8867 0.00167669 0.0016943 2026-02-22T23:43:02.341 INFO:tasks.workunit.client.0.trial122.stdout: 13 16 122700 122684 36.8614 36.9414 0.00148004 0.00169396 2026-02-22T23:43:03.341 INFO:tasks.workunit.client.0.trial122.stdout: 14 16 132219 132203 36.8842 37.1836 0.00134843 0.00169293 2026-02-22T23:43:04.341 INFO:tasks.workunit.client.0.trial122.stdout: 15 16 135828 135812 35.365 14.0977 0.0740311 0.00176096 2026-02-22T23:43:05.341 INFO:tasks.workunit.client.0.trial122.stdout: 16 16 144498 144482 35.2713 33.8672 0.001153 0.00177043 2026-02-22T23:43:06.341 INFO:tasks.workunit.client.0.trial122.stdout: 17 16 154326 154310 35.4546 38.3906 0.00149546 0.00176129 2026-02-22T23:43:07.341 INFO:tasks.workunit.client.0.trial122.stdout: 18 16 164150 164134 35.6166 38.375 0.00201962 0.0017533 2026-02-22T23:43:08.341 INFO:tasks.workunit.client.0.trial122.stdout: 19 15 173852 173837 35.7368 37.9023 0.00191768 0.00174739 2026-02-22T23:43:09.341 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:43:09.344146+0000 min lat: 0.000475142 max lat: 0.125965 avg lat: 0.00174307 2026-02-22T23:43:09.342 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:43:09.342 INFO:tasks.workunit.client.0.trial122.stdout: 20 16 183457 183441 35.8256 37.5156 0.00176278 0.00174307 2026-02-22T23:43:10.341 INFO:tasks.workunit.client.0.trial122.stdout: 21 16 185887 185871 34.5716 9.49219 0.224838 0.0017986 2026-02-22T23:43:11.341 INFO:tasks.workunit.client.0.trial122.stdout: 22 16 185999 185983 33.02 0.4375 0.00218073 0.00188778 2026-02-22T23:43:12.341 INFO:tasks.workunit.client.0.trial122.stdout: 23 15 195307 195292 33.1653 36.3633 0.000989538 0.001883 2026-02-22T23:43:13.342 INFO:tasks.workunit.client.0.trial122.stdout: 24 16 204787 204771 33.3261 37.0273 0.0014455 0.00187391 2026-02-22T23:43:14.342 INFO:tasks.workunit.client.0.trial122.stdout: 25 16 214352 214336 33.4874 37.3633 0.00185188 0.00186486 2026-02-22T23:43:15.342 INFO:tasks.workunit.client.0.trial122.stdout: 26 16 223864 223848 33.6284 37.1562 0.00156005 0.00185708 2026-02-22T23:43:16.342 INFO:tasks.workunit.client.0.trial122.stdout: 27 16 233345 233329 33.7545 37.0352 0.00137553 0.00185007 2026-02-22T23:43:17.342 INFO:tasks.workunit.client.0.trial122.stdout: 28 16 242881 242865 33.8792 37.25 0.00219705 0.00184328 2026-02-22T23:43:18.342 INFO:tasks.workunit.client.0.trial122.stdout: 29 16 252375 252359 33.9897 37.0859 0.00202148 0.00183731 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Total time run: 30.0014 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Total writes made: 261934 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Write size: 4096 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Object size: 4096 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Bandwidth (MB/sec): 34.1044 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Bandwidth: 9.75205 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Max bandwidth (MB/sec): 41.6406 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Min bandwidth (MB/sec): 0.4375 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Average IOPS: 8730 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Stddev IOPS: 2496.52 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Max IOPS: 10660 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Min IOPS: 112 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Average Latency(s): 0.00183113 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Latency(s): 0.00464606 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Max latency(s): 0.23936 2026-02-22T23:43:19.342 INFO:tasks.workunit.client.0.trial122.stdout:Min latency(s): 0.000475142 2026-02-22T23:43:19.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:339: TEST_bluestore: wait_for_clean 2026-02-22T23:43:19.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:43:19.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:43:19.345 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:43:19.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:43:19.346 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:43:19.346 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:43:19.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:43:19.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:43:19.346 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:43:19.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:43:19.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:43:19.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:43:19.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:43:19.439 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:43:19.440 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids='0 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr:1 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr:2 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr:3' 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:43:19.633 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:43:19.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=304942678032 2026-02-22T23:43:19.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 304942678032 2026-02-22T23:43:19.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-304942678032' 2026-02-22T23:43:19.710 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:43:19.711 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-22T23:43:19.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=317827579917 2026-02-22T23:43:19.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 317827579917 2026-02-22T23:43:19.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-304942678032 1-317827579917' 2026-02-22T23:43:19.785 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:43:19.786 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-22T23:43:19.860 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=335007449099 2026-02-22T23:43:19.860 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 335007449099 2026-02-22T23:43:19.860 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-304942678032 1-317827579917 2-335007449099' 2026-02-22T23:43:19.860 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:43:19.861 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-22T23:43:19.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=347892350985 2026-02-22T23:43:19.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 347892350985 2026-02-22T23:43:19.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-304942678032 1-317827579917 2-335007449099 3-347892350985' 2026-02-22T23:43:19.936 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:43:19.937 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-304942678032 2026-02-22T23:43:19.937 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:43:19.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:43:19.940 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-304942678032 2026-02-22T23:43:19.940 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:43:19.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=304942678032 2026-02-22T23:43:19.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 304942678032' 2026-02-22T23:43:19.941 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 304942678032 2026-02-22T23:43:19.942 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:43:20.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678030 -lt 304942678032 2026-02-22T23:43:20.132 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:43:21.133 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:43:21.134 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:43:21.331 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678030 -lt 304942678032 2026-02-22T23:43:21.331 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:43:22.332 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-22T23:43:22.332 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:43:23.043 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 304942678032 -lt 304942678032 2026-02-22T23:43:23.044 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:43:23.044 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 1-317827579917 2026-02-22T23:43:23.045 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:43:23.046 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=1 2026-02-22T23:43:23.047 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 1-317827579917 2026-02-22T23:43:23.047 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:43:23.048 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=317827579917 2026-02-22T23:43:23.048 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.1 seq 317827579917' 2026-02-22T23:43:23.049 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.1 seq 317827579917 2026-02-22T23:43:23.049 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-22T23:43:23.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 317827579918 -lt 317827579917 2026-02-22T23:43:23.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:43:23.694 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 2-335007449099 2026-02-22T23:43:23.694 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:43:23.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=2 2026-02-22T23:43:23.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 2-335007449099 2026-02-22T23:43:23.696 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:43:23.698 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=335007449099 2026-02-22T23:43:23.698 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.2 seq 335007449099 2026-02-22T23:43:23.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.2 seq 335007449099' 2026-02-22T23:43:23.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-22T23:43:24.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 335007449099 -lt 335007449099 2026-02-22T23:43:24.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:43:24.028 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 3-347892350985 2026-02-22T23:43:24.028 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:43:24.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=3 2026-02-22T23:43:24.030 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 3-347892350985 2026-02-22T23:43:24.030 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:43:24.032 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=347892350985 2026-02-22T23:43:24.032 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.3 seq 347892350985 2026-02-22T23:43:24.032 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.3 seq 347892350985' 2026-02-22T23:43:24.033 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-22T23:43:24.445 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 347892350985 -lt 347892350985 2026-02-22T23:43:24.446 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:43:24.447 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:43:24.447 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:43:24.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-22T23:43:24.934 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:43:24.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:43:24.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:43:24.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:43:24.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:43:24.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:43:24.935 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:43:25.135 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-22T23:43:25.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:43:25.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:43:25.136 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:43:25.404 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-22T23:43:25.405 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:43:25.405 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:43:25.405 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:43:25.405 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:43:25.405 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:43:25.714 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:43:25.715 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:43:25.715 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:43:25.716 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:43:25.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:43:25.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:43:25.718 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:43:25.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:43:25.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:43:25.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:43:25.720 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:43:25.721 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:43:25.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:43:25.722 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-22T23:43:31.690 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:43:31.690 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.690 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92291 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:43:31.692 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-22T23:43:31.693 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:43:31.693 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:43:31.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:43:31.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:43:31.693 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:43:31.695 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:43:31.695 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:43:31.696 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:43:31.697 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:43:31.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:43:31.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:43:31.699 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:43:31.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:43:31.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:43:31.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:43:31.701 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:43:31.702 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:43:31.703 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:43:31.703 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-22T23:43:31.704 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:43:31.704 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.704 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.704 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92291 2026-02-22T23:43:31.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:43:31.705 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:43:31.706 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-22T23:43:31.707 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:43:31.707 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.707 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.707 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92291 2026-02-22T23:43:31.708 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:43:31.708 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-22T23:43:31.708 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore2 td/osd-bluefs-volume-ops 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:343: TEST_bluestore2: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: ulimit -n 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:345: TEST_bluestore2: local flimit=4096 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:346: TEST_bluestore2: '[' 4096 -lt 1536 ']' 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: export CEPH_MON=127.0.0.1:7146 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:349: TEST_bluestore2: CEPH_MON=127.0.0.1:7146 2026-02-22T23:43:31.709 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:350: TEST_bluestore2: export CEPH_ARGS 2026-02-22T23:43:31.710 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: uuidgen 2026-02-22T23:43:31.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:351: TEST_bluestore2: CEPH_ARGS+='--fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none ' 2026-02-22T23:43:31.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:352: TEST_bluestore2: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-22T23:43:31.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:353: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-22T23:43:31.711 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:354: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:355: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:356: TEST_bluestore2: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:357: TEST_bluestore2: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:358: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:359: TEST_bluestore2: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:360: TEST_bluestore2: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:362: TEST_bluestore2: run_mon td/osd-bluefs-volume-ops a 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-22T23:43:31.712 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:31.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:43:31.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:43:31.734 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:43:31.735 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:43:31.735 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.735 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.735 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:43:31.735 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:43:31.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:43:31.766 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:43:31.766 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:43:31.766 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:43:31.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:43:31.767 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:43:31.767 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:43:31.767 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:43:31.767 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:43:31.768 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:43:31.768 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.768 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.768 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92291/ceph-mon.a.asok 2026-02-22T23:43:31.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:43:31.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92291/ceph-mon.a.asok config get fsid 2026-02-22T23:43:31.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:43:31.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:43:31.822 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:43:31.823 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:43:31.823 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:43:31.823 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:43:31.823 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:43:31.823 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:43:31.824 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:43:31.824 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.824 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.824 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92291/ceph-mon.a.asok 2026-02-22T23:43:31.824 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:43:31.824 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92291/ceph-mon.a.asok config get mon_host 2026-02-22T23:43:31.877 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:363: TEST_bluestore2: run_mgr td/osd-bluefs-volume-ops x 2026-02-22T23:43:31.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:31.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:43:31.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:43:31.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:43:31.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-22T23:43:31.878 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:31.988 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:43:31.989 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:43:31.990 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:364: TEST_bluestore2: run_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:43:32.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:43:32.012 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:43:32.013 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:43:32.014 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:43:32.014 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=e47b9e53-b9d3-4caf-a37f-6ba284f19a49 2026-02-22T23:43:32.015 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 e47b9e53-b9d3-4caf-a37f-6ba284f19a49 2026-02-22T23:43:32.015 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 e47b9e53-b9d3-4caf-a37f-6ba284f19a49' 2026-02-22T23:43:32.015 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:43:32.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQCklJtpitbGARAA3fwwyW6O1l78Jx22JEiUrw== 2026-02-22T23:43:32.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQCklJtpitbGARAA3fwwyW6O1l78Jx22JEiUrw=="}' 2026-02-22T23:43:32.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new e47b9e53-b9d3-4caf-a37f-6ba284f19a49 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-22T23:43:32.141 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:43:32.149 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-22T23:43:32.151 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCklJtpitbGARAA3fwwyW6O1l78Jx22JEiUrw== --osd-uuid e47b9e53-b9d3-4caf-a37f-6ba284f19a49 2026-02-22T23:43:32.168 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:32.166+0000 7fc1e40548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:43:32.170 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:32.170+0000 7fc1e40548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:43:32.171 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:32.170+0000 7fc1e40548c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:43:32.171 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:32.170+0000 7fc1e40548c0 -1 bdev(0x55b489e18c00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:43:32.171 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:32.170+0000 7fc1e40548c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-22T23:43:37.530 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-22T23:43:37.530 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:43:37.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:43:37.531 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:43:37.532 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:43:37.810 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:43:37.810 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:43:37.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:43:37.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:43:37.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:43:37.811 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:43:37.828 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:37.826+0000 7f9046ed48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:43:37.829 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:37.830+0000 7f9046ed48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:43:37.830 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:37.830+0000 7f9046ed48c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:43:38.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:43:38.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:43:38.010 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:43:38.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:43:38.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:43:38.011 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:38.011 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:43:38.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:43:38.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:38.012 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:38.206 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:43:39.207 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:43:39.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:39.208 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:43:39.208 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:43:39.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:39.209 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:39.416 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:43:39.591 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:39.590+0000 7f9046ed48c0 -1 Falling back to public interface 2026-02-22T23:43:40.417 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:43:40.418 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:40.418 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:43:40.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:43:40.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:40.419 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:40.616 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:43:41.617 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:43:41.618 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:41.618 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:43:41.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:43:41.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:41.619 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:41.816 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:43:42.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:43:42.818 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:42.818 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:43:42.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:43:42.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:42.819 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:43.027 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:43:44.028 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:43:44.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:44.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:43:44.029 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:43:44.029 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:44.030 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:44.170 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:43:44.170+0000 7f9046ed48c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:43:44.233 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:43:45.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:43:45.235 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:43:45.235 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:43:45.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:43:45.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:43:45.236 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:43:45.431 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3129204740,v1:127.0.0.1:6803/3129204740] [v2:127.0.0.1:6804/3129204740,v1:127.0.0.1:6805/3129204740] exists,up e47b9e53-b9d3-4caf-a37f-6ba284f19a49 2026-02-22T23:43:45.432 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:43:45.432 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:43:45.432 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:43:45.432 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:43:45.433 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:365: TEST_bluestore2: osd_pid0=140516 2026-02-22T23:43:45.433 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:367: TEST_bluestore2: sleep 5 2026-02-22T23:43:50.434 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:368: TEST_bluestore2: create_pool foo 16 2026-02-22T23:43:50.435 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-22T23:43:50.689 INFO:tasks.workunit.client.0.trial122.stderr:pool 'foo' already exists 2026-02-22T23:43:50.699 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:43:51.700 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:370: TEST_bluestore2: retry = 0 2026-02-22T23:43:51.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 370: retry: command not found 2026-02-22T23:43:51.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:371: TEST_bluestore2: [[ '' -le 5 ]] 2026-02-22T23:43:51.701 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:373: TEST_bluestore2: timeout 60 rados bench -p foo 10 write --write-omap --no-cleanup 2026-02-22T23:43:51.723 INFO:tasks.workunit.client.0.trial122.stdout:hints = 1 2026-02-22T23:43:51.724 INFO:tasks.workunit.client.0.trial122.stdout:Maintaining 16 concurrent writes of 4194304 bytes to objects of size 4194304 for up to 10 seconds or 0 objects 2026-02-22T23:43:51.724 INFO:tasks.workunit.client.0.trial122.stdout:Object prefix: benchmark_data_trial122_143140 2026-02-22T23:43:51.761 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:43:51.761 INFO:tasks.workunit.client.0.trial122.stdout: 0 0 0 0 0 0 - 0 2026-02-22T23:43:52.761 INFO:tasks.workunit.client.0.trial122.stdout: 1 16 182 166 663.957 664 0.088821 0.0945601 2026-02-22T23:43:53.761 INFO:tasks.workunit.client.0.trial122.stdout: 2 16 374 358 715.947 768 0.0685431 0.0867901 2026-02-22T23:43:54.761 INFO:tasks.workunit.client.0.trial122.stdout: 3 16 582 566 754.608 832 0.0809377 0.0834521 2026-02-22T23:43:55.761 INFO:tasks.workunit.client.0.trial122.stdout: 4 16 774 758 757.939 768 0.0857346 0.0829817 2026-02-22T23:43:56.761 INFO:tasks.workunit.client.0.trial122.stdout: 5 16 998 982 785.536 896 0.0724553 0.0806419 2026-02-22T23:43:57.761 INFO:tasks.workunit.client.0.trial122.stdout: 6 16 1190 1174 782.603 768 0.0702055 0.0813107 2026-02-22T23:43:58.761 INFO:tasks.workunit.client.0.trial122.stdout: 7 16 1382 1366 780.508 768 0.0797007 0.0814704 2026-02-22T23:43:59.762 INFO:tasks.workunit.client.0.trial122.stdout: 8 16 1601 1585 792.433 876 0.067678 0.0805371 2026-02-22T23:44:00.762 INFO:tasks.workunit.client.0.trial122.stdout: 9 16 1782 1766 784.822 724 0.0862153 0.0811615 2026-02-22T23:44:01.762 INFO:tasks.workunit.client.0.trial122.stdout: 10 16 1958 1942 776.734 704 0.0848851 0.0821583 2026-02-22T23:44:02.762 INFO:tasks.workunit.client.0.trial122.stdout:Total time run: 10.0776 2026-02-22T23:44:02.762 INFO:tasks.workunit.client.0.trial122.stdout:Total writes made: 1958 2026-02-22T23:44:02.762 INFO:tasks.workunit.client.0.trial122.stdout:Write size: 4194304 2026-02-22T23:44:02.762 INFO:tasks.workunit.client.0.trial122.stdout:Object size: 4194304 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Bandwidth (MB/sec): 777.167 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Bandwidth: 73.195 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Max bandwidth (MB/sec): 896 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Min bandwidth (MB/sec): 664 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Average IOPS: 194 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Stddev IOPS: 18.2988 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Max IOPS: 224 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Min IOPS: 166 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Average Latency(s): 0.0822117 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Latency(s): 0.0116789 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Max latency(s): 0.121156 2026-02-22T23:44:02.763 INFO:tasks.workunit.client.0.trial122.stdout:Min latency(s): 0.0365482 2026-02-22T23:44:02.775 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:376: TEST_bluestore2: sleep 10 2026-02-22T23:44:12.778 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-22T23:44:12.778 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: jq .bluefs.db_used_bytes 2026-02-22T23:44:12.862 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:378: TEST_bluestore2: db_used=783286272 2026-02-22T23:44:12.863 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: ceph tell osd.0 perf dump bluefs 2026-02-22T23:44:12.863 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: jq .bluefs.slow_used_bytes 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:379: TEST_bluestore2: spilled_over=66781184 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:380: TEST_bluestore2: (( retry+=1 )) 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: test 66781184 -eq 0 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:381: TEST_bluestore2: break 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:383: TEST_bluestore2: test 66781184 -gt 0 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 140516 2026-02-22T23:44:12.941 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: sleep 1 2026-02-22T23:44:13.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:385: TEST_bluestore2: kill 140516 2026-02-22T23:44:13.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 385: kill: (140516) - No such process 2026-02-22T23:44:13.942 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:386: TEST_bluestore2: ceph osd down 0 2026-02-22T23:44:14.137 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:44:14.146 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:388: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --devs-source td/osd-bluefs-volume-ops/0/block.db --dev-target td/osd-bluefs-volume-ops/0/block --command bluefs-bdev-migrate 2026-02-22T23:44:14.165 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:44:17.297 INFO:tasks.workunit.client.0.trial122.stdout: device removed:1 td/osd-bluefs-volume-ops/0/block.db 2026-02-22T23:44:17.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:393: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command bluefs-bdev-sizes 2026-02-22T23:44:17.334 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:44:18.152 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x100000000(4 GiB) : using 0x30cc6000(781 MiB) 2026-02-22T23:44:18.406 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:396: TEST_bluestore2: ceph-bluestore-tool --path td/osd-bluefs-volume-ops/0 --command fsck 2026-02-22T23:44:20.761 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:44:20.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:399: TEST_bluestore2: activate_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:44:20.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:20.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:44:20.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:44:20.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:20.848 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:44:20.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:44:20.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:44:20.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:44:20.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:44:20.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:44:20.850 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:44:20.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:44:20.851 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:44:20.852 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-22T23:44:20.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:44:20.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:44:20.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:44:20.852 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:44:20.869 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:20.869+0000 7fdce08028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:20.870 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:20.869+0000 7fdce08028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:20.871 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:20.869+0000 7fdce08028c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:21.056 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:44:21.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:44:21.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:21.057 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:21.252 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:22.159 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:22.157+0000 7fdce08028c0 -1 Falling back to public interface 2026-02-22T23:44:22.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:22.253 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:22.253 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:44:22.254 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:44:22.254 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:22.254 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:22.446 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:23.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:23.448 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:23.448 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:44:23.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:44:23.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:23.449 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:23.648 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:24.649 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:24.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:24.650 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:44:24.650 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:44:24.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:24.651 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:24.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:25.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:25.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:25.847 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:44:25.847 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:44:25.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:25.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:26.059 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:27.061 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:27.061 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:27.061 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:44:27.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:44:27.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:27.062 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:27.256 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:27.333 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:27.333+0000 7fdce08028c0 -1 osd.0 8 log_to_monitors true 2026-02-22T23:44:28.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:28.257 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:28.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:44:28.258 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:44:28.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:28.258 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:28.461 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 11 up_thru 11 down_at 9 last_clean_interval [5,8) [v2:127.0.0.1:6802/330036343,v1:127.0.0.1:6803/330036343] [v2:127.0.0.1:6804/330036343,v1:127.0.0.1:6805/330036343] exists,up e47b9e53-b9d3-4caf-a37f-6ba284f19a49 2026-02-22T23:44:28.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:44:28.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:44:28.462 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:44:28.462 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:44:28.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:400: TEST_bluestore2: osd_pid0=144376 2026-02-22T23:44:28.463 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:402: TEST_bluestore2: wait_for_clean 2026-02-22T23:44:28.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:44:28.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:44:28.464 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:44:28.465 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:44:28.465 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:44:28.465 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:44:28.465 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:44:28.465 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:44:28.465 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:44:28.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:44:28.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:44:28.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:44:28.559 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:44:28.560 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:44:28.560 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:44:28.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-22T23:44:28.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:44:28.767 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:44:28.768 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:44:28.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=47244640259 2026-02-22T23:44:28.844 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 47244640259 2026-02-22T23:44:28.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-47244640259' 2026-02-22T23:44:28.845 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:44:28.845 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-47244640259 2026-02-22T23:44:28.845 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:44:28.846 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:44:28.847 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-47244640259 2026-02-22T23:44:28.847 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:44:28.848 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=47244640259 2026-02-22T23:44:28.848 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 47244640259 2026-02-22T23:44:28.849 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 47244640259' 2026-02-22T23:44:28.849 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:44:29.059 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 47244640259 2026-02-22T23:44:29.059 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:44:30.060 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:44:30.060 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:44:30.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 1 -lt 47244640259 2026-02-22T23:44:30.269 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:44:31.270 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-22T23:44:31.271 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:44:31.470 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 47244640259 -lt 47244640259 2026-02-22T23:44:31.470 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:44:31.471 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:44:31.471 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:44:31.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-22T23:44:31.725 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:44:31.726 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:44:31.726 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:44:31.726 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:44:31.726 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:44:31.726 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:44:31.727 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:44:31.929 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-22T23:44:31.930 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:44:31.930 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:44:31.931 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:15: run: teardown td/osd-bluefs-volume-ops 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:44:32.198 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:44:32.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:44:32.199 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:44:32.305 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:44:32.305 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:44:32.306 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:44:32.307 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:44:32.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:44:32.308 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:44:32.308 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:44:32.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:44:32.309 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:44:32.310 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:44:32.310 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:44:32.311 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:44:32.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:44:32.312 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-22T23:44:32.472 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:44:32.473 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.473 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.473 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92291 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:12: run: for func in $funcs 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:13: run: setup td/osd-bluefs-volume-ops 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:131: setup: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:132: setup: teardown td/osd-bluefs-volume-ops 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs= 2026-02-22T23:44:32.475 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-22T23:44:32.476 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:44:32.476 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:44:32.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:44:32.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:44:32.476 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:44:32.477 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:44:32.478 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:44:32.479 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:44:32.479 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:44:32.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:44:32.480 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:44:32.480 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:44:32.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:44:32.481 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:44:32.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:44:32.482 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:44:32.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:44:32.485 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o '' = 1 ']' 2026-02-22T23:44:32.485 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-22T23:44:32.486 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:44:32.486 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.486 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.487 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92291 2026-02-22T23:44:32.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:44:32.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:44:32.488 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:133: setup: mkdir -p td/osd-bluefs-volume-ops 2026-02-22T23:44:32.489 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: get_asok_dir 2026-02-22T23:44:32.489 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.489 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.489 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:134: setup: mkdir -p /tmp/ceph-asok.92291 2026-02-22T23:44:32.490 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: ulimit -n 2026-02-22T23:44:32.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:135: setup: '[' 4096 -le 1024 ']' 2026-02-22T23:44:32.491 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:138: setup: '[' -z '' ']' 2026-02-22T23:44:32.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:139: setup: trap 'teardown td/osd-bluefs-volume-ops 1' TERM HUP INT 2026-02-22T23:44:32.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: TEST_bluestore_expand td/osd-bluefs-volume-ops 2026-02-22T23:44:32.492 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:406: TEST_bluestore_expand: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.492 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: ulimit -n 2026-02-22T23:44:32.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:408: TEST_bluestore_expand: local flimit=4096 2026-02-22T23:44:32.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:409: TEST_bluestore_expand: '[' 4096 -lt 1536 ']' 2026-02-22T23:44:32.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: export CEPH_MON=127.0.0.1:7146 2026-02-22T23:44:32.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:412: TEST_bluestore_expand: CEPH_MON=127.0.0.1:7146 2026-02-22T23:44:32.493 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:413: TEST_bluestore_expand: export CEPH_ARGS 2026-02-22T23:44:32.493 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: uuidgen 2026-02-22T23:44:32.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:414: TEST_bluestore_expand: CEPH_ARGS+='--fsid=9f06c412-45f0-4208-90c5-2f5dfcdd1a2d --auth-supported=none ' 2026-02-22T23:44:32.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:415: TEST_bluestore_expand: CEPH_ARGS+='--mon-host=127.0.0.1:7146 ' 2026-02-22T23:44:32.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:416: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_size=4294967296 ' 2026-02-22T23:44:32.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:417: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_create=true ' 2026-02-22T23:44:32.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:418: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_db_size=1073741824 ' 2026-02-22T23:44:32.495 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:419: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_block_wal_create=false ' 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:420: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_fsck_on_mount=true ' 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:421: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_size=1 ' 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:422: TEST_bluestore_expand: CEPH_ARGS+='--osd_pool_default_min_size=1 ' 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:423: TEST_bluestore_expand: CEPH_ARGS+='--bluestore_debug_enforce_settings=ssd ' 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:425: TEST_bluestore_expand: run_mon td/osd-bluefs-volume-ops a 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:448: run_mon: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:449: run_mon: shift 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:450: run_mon: local id=a 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:451: run_mon: shift 2026-02-22T23:44:32.496 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:452: run_mon: local data=td/osd-bluefs-volume-ops/a 2026-02-22T23:44:32.497 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:455: run_mon: ceph-mon --id a --mkfs --mon-data=td/osd-bluefs-volume-ops/a --run-dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.518 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: get_asok_path 2026-02-22T23:44:32.518 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:44:32.518 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:44:32.519 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:44:32.519 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.519 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.519 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:44:32.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:462: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/osd-bluefs-volume-ops/a '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --mon-cluster-log-file=td/osd-bluefs-volume-ops/log --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-22T23:44:32.549 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: cat 2026-02-22T23:44:32.550 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a fsid 2026-02-22T23:44:32.550 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:44:32.550 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:44:32.550 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=fsid 2026-02-22T23:44:32.551 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .fsid 2026-02-22T23:44:32.551 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92291/ceph-mon.a.asok 2026-02-22T23:44:32.552 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:44:32.553 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92291/ceph-mon.a.asok config get fsid 2026-02-22T23:44:32.606 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:487: run_mon: get_config mon a mon_host 2026-02-22T23:44:32.606 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1125: get_config: local daemon=mon 2026-02-22T23:44:32.607 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1126: get_config: local id=a 2026-02-22T23:44:32.607 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1127: get_config: local config=mon_host 2026-02-22T23:44:32.607 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1132: get_config: jq -r .mon_host 2026-02-22T23:44:32.608 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: get_asok_path mon.a 2026-02-22T23:44:32.608 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name=mon.a 2026-02-22T23:44:32.608 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n mon.a ']' 2026-02-22T23:44:32.608 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: get_asok_dir 2026-02-22T23:44:32.609 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.609 INFO:tasks.workunit.client.0.trial122.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.609 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:118: get_asok_path: echo /tmp/ceph-asok.92291/ceph-mon.a.asok 2026-02-22T23:44:32.609 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: CEPH_ARGS= 2026-02-22T23:44:32.609 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1129: get_config: ceph --format json daemon /tmp/ceph-asok.92291/ceph-mon.a.asok config get mon_host 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:426: TEST_bluestore_expand: run_mgr td/osd-bluefs-volume-ops x 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:553: run_mgr: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:554: run_mgr: shift 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:555: run_mgr: local id=x 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:556: run_mgr: shift 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:557: run_mgr: local data=td/osd-bluefs-volume-ops/x 2026-02-22T23:44:32.662 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:559: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-22T23:44:32.773 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: get_asok_path 2026-02-22T23:44:32.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:44:32.774 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:44:32.774 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:44:32.774 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.774 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:44:32.775 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:44:32.776 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:561: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/osd-bluefs-volume-ops/x '--log-file=td/osd-bluefs-volume-ops/$name.log' '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --run-dir=td/osd-bluefs-volume-ops '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-22T23:44:32.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:427: TEST_bluestore_expand: run_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:44:32.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:44:32.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: shift 2026-02-22T23:44:32.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: local id=0 2026-02-22T23:44:32.799 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: shift 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=9f06c412-45f0-4208-90c5-2f5dfcdd1a2d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --chdir=' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+= 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: get_asok_path 2026-02-22T23:44:32.800 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=' --debug-ms=1' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: ceph_args+=' --debug-monc=20' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:652: run_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:44:32.801 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: ceph_args+=' ' 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph_args+= 2026-02-22T23:44:32.802 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:660: run_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:44:32.803 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: uuidgen 2026-02-22T23:44:32.803 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: local uuid=310a5282-36f1-4995-8a1f-dc926e116260 2026-02-22T23:44:32.803 INFO:tasks.workunit.client.0.trial122.stdout:add osd0 310a5282-36f1-4995-8a1f-dc926e116260 2026-02-22T23:44:32.804 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:663: run_osd: echo 'add osd0 310a5282-36f1-4995-8a1f-dc926e116260' 2026-02-22T23:44:32.804 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: ceph-authtool --gen-print-key 2026-02-22T23:44:32.816 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:664: run_osd: OSD_SECRET=AQDglJtpkXHBMBAADhtku3jZNLv8Eyp8zgEegg== 2026-02-22T23:44:32.816 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:665: run_osd: echo '{"cephx_secret": "AQDglJtpkXHBMBAADhtku3jZNLv8Eyp8zgEegg=="}' 2026-02-22T23:44:32.816 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: ceph osd new 310a5282-36f1-4995-8a1f-dc926e116260 -i td/osd-bluefs-volume-ops/0/new.json 2026-02-22T23:44:32.931 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:44:32.939 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: rm td/osd-bluefs-volume-ops/0/new.json 2026-02-22T23:44:32.940 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=9f06c412-45f0-4208-90c5-2f5dfcdd1a2d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDglJtpkXHBMBAADhtku3jZNLv8Eyp8zgEegg== --osd-uuid 310a5282-36f1-4995-8a1f-dc926e116260 2026-02-22T23:44:32.959 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:32.957+0000 7ff7142c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:32.960 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:32.957+0000 7ff7142c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:32.961 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:32.961+0000 7ff7142c88c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:32.962 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:32.961+0000 7ff7142c88c0 -1 bdev(0x55a03997ac00 td/osd-bluefs-volume-ops/0/block) open stat got: (1) Operation not permitted 2026-02-22T23:44:32.962 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:32.961+0000 7ff7142c88c0 -1 bluestore(td/osd-bluefs-volume-ops/0) _read_fsid unparsable uuid 2026-02-22T23:44:38.078 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:670: run_osd: local key_fn=td/osd-bluefs-volume-ops/0/keyring 2026-02-22T23:44:38.079 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:671: run_osd: cat 2026-02-22T23:44:38.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: echo adding osd0 key to auth repository 2026-02-22T23:44:38.080 INFO:tasks.workunit.client.0.trial122.stdout:adding osd0 key to auth repository 2026-02-22T23:44:38.080 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:676: run_osd: ceph -i td/osd-bluefs-volume-ops/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-22T23:44:38.355 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:44:38.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:677: run_osd: echo start osd.0 2026-02-22T23:44:38.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:678: run_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=9f06c412-45f0-4208-90c5-2f5dfcdd1a2d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:44:38.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: ceph osd dump --format=json 2026-02-22T23:44:38.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: jq '.flags_set[]' 2026-02-22T23:44:38.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:681: run_osd: grep -q '"noup"' 2026-02-22T23:44:38.386 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:38.385+0000 7fe7e73af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:38.388 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:38.385+0000 7fe7e73af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:38.389 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:38.389+0000 7fe7e73af8c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:44:38.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:684: run_osd: wait_for_osd up 0 2026-02-22T23:44:38.563 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:44:38.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:44:38.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:44:38.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:44:38.564 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:38.564 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:44:38.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:44:38.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:38.565 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:38.763 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:39.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:39.764 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:39.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:44:39.765 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:44:39.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:39.765 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:39.972 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:40.138 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:40.137+0000 7fe7e73af8c0 -1 Falling back to public interface 2026-02-22T23:44:40.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:40.974 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:40.974 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:44:40.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:44:40.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:40.975 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:41.176 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:42.177 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:42.178 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:42.178 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:44:42.179 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:44:42.179 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:42.179 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:42.381 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:43.382 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:43.383 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:43.383 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:44:43.383 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:44:43.384 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:43.384 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:43.589 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:44.590 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:44.591 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:44:44.591 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:44:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:44.592 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:44.741 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:44:44.741+0000 7fe7e73af8c0 -1 osd.0 0 log_to_monitors true 2026-02-22T23:44:44.792 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:44:45.794 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:44:45.794 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:44:45.794 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:44:45.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:44:45.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:44:45.795 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:44:45.999 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/4219709147,v1:127.0.0.1:6803/4219709147] [v2:127.0.0.1:6804/4219709147,v1:127.0.0.1:6805/4219709147] exists,up 310a5282-36f1-4995-8a1f-dc926e116260 2026-02-22T23:44:45.999 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:44:46.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:44:46.000 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:44:46.000 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:44:46.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:428: TEST_bluestore_expand: osd_pid0=147135 2026-02-22T23:44:46.001 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:430: TEST_bluestore_expand: sleep 5 2026-02-22T23:44:51.002 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:431: TEST_bluestore_expand: create_pool foo 16 2026-02-22T23:44:51.003 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:541: create_pool: ceph osd pool create foo 16 2026-02-22T23:44:51.272 INFO:tasks.workunit.client.0.trial122.stderr:pool 'foo' already exists 2026-02-22T23:44:51.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:542: create_pool: sleep 1 2026-02-22T23:44:52.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:434: TEST_bluestore_expand: timeout 60 rados bench -p foo 30 write -b 4096 --no-cleanup 2026-02-22T23:44:52.308 INFO:tasks.workunit.client.0.trial122.stdout:hints = 1 2026-02-22T23:44:52.308 INFO:tasks.workunit.client.0.trial122.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 30 seconds or 0 objects 2026-02-22T23:44:52.308 INFO:tasks.workunit.client.0.trial122.stdout:Object prefix: benchmark_data_trial122_149107 2026-02-22T23:44:52.308 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:44:52.308 INFO:tasks.workunit.client.0.trial122.stdout: 0 0 0 0 0 0 - 0 2026-02-22T23:44:53.308 INFO:tasks.workunit.client.0.trial122.stdout: 1 16 14148 14132 55.1991 55.2031 0.00136743 0.00113049 2026-02-22T23:44:54.308 INFO:tasks.workunit.client.0.trial122.stdout: 2 16 28358 28342 55.3514 55.5078 0.00103317 0.00112786 2026-02-22T23:44:55.308 INFO:tasks.workunit.client.0.trial122.stdout: 3 16 42462 42446 55.2641 55.0938 0.0011462 0.00112981 2026-02-22T23:44:56.308 INFO:tasks.workunit.client.0.trial122.stdout: 4 16 56438 56422 55.0955 54.5938 0.00117406 0.00113326 2026-02-22T23:44:57.308 INFO:tasks.workunit.client.0.trial122.stdout: 5 16 70471 70455 55.0389 54.8164 0.00112497 0.00113445 2026-02-22T23:44:58.308 INFO:tasks.workunit.client.0.trial122.stdout: 6 15 84507 84492 55.0037 54.832 0.00110179 0.00113521 2026-02-22T23:44:59.308 INFO:tasks.workunit.client.0.trial122.stdout: 7 16 98830 98814 55.1376 55.9453 0.00141902 0.00113248 2026-02-22T23:45:00.308 INFO:tasks.workunit.client.0.trial122.stdout: 8 16 112283 112267 54.8137 52.5508 0.0010887 0.00113916 2026-02-22T23:45:01.308 INFO:tasks.workunit.client.0.trial122.stdout: 9 16 125083 125067 54.2784 50 0.00149107 0.00115041 2026-02-22T23:45:02.308 INFO:tasks.workunit.client.0.trial122.stdout: 10 16 137854 137838 53.8389 49.8867 0.000967588 0.00115983 2026-02-22T23:45:03.308 INFO:tasks.workunit.client.0.trial122.stdout: 11 16 150450 150434 53.4171 49.2031 0.00184785 0.00116903 2026-02-22T23:45:04.308 INFO:tasks.workunit.client.0.trial122.stdout: 12 16 163098 163082 53.0826 49.4062 0.00142552 0.00117639 2026-02-22T23:45:05.309 INFO:tasks.workunit.client.0.trial122.stdout: 13 16 175218 175202 52.6408 47.3438 0.0012837 0.00118628 2026-02-22T23:45:06.309 INFO:tasks.workunit.client.0.trial122.stdout: 14 15 187553 187538 52.3225 48.1875 0.000935489 0.0011935 2026-02-22T23:45:07.309 INFO:tasks.workunit.client.0.trial122.stdout: 15 16 199813 199797 52.0265 47.8867 0.00133525 0.00120032 2026-02-22T23:45:08.309 INFO:tasks.workunit.client.0.trial122.stdout: 16 16 212470 212454 51.8647 49.4414 0.00088743 0.00120407 2026-02-22T23:45:09.309 INFO:tasks.workunit.client.0.trial122.stdout: 17 16 224863 224847 51.6613 48.4102 0.00121427 0.00120883 2026-02-22T23:45:10.309 INFO:tasks.workunit.client.0.trial122.stdout: 18 16 236988 236972 51.4223 47.3633 0.00116568 0.00121444 2026-02-22T23:45:11.309 INFO:tasks.workunit.client.0.trial122.stdout: 19 16 249256 249240 51.2379 47.9219 0.001307 0.0012188 2026-02-22T23:45:12.309 INFO:tasks.workunit.client.0.trial122.stdout:2026-02-22T23:45:12.312121+0000 min lat: 0.000518624 max lat: 0.00784385 avg lat: 0.00122339 2026-02-22T23:45:12.309 INFO:tasks.workunit.client.0.trial122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-22T23:45:12.309 INFO:tasks.workunit.client.0.trial122.stdout: 20 16 261394 261378 51.0465 47.4141 0.00109982 0.00122339 2026-02-22T23:45:13.309 INFO:tasks.workunit.client.0.trial122.stdout: 21 16 273527 273511 50.8724 47.3945 0.00159333 0.00122758 2026-02-22T23:45:14.309 INFO:tasks.workunit.client.0.trial122.stdout: 22 16 285595 285579 50.7026 47.1406 0.000773779 0.0012317 2026-02-22T23:45:15.309 INFO:tasks.workunit.client.0.trial122.stdout: 23 16 297693 297677 50.5527 47.2578 0.00119569 0.00123535 2026-02-22T23:45:16.309 INFO:tasks.workunit.client.0.trial122.stdout: 24 16 309738 309722 50.4067 47.0508 0.00129053 0.00123895 2026-02-22T23:45:17.309 INFO:tasks.workunit.client.0.trial122.stdout: 25 16 322019 322003 50.3091 47.9727 0.000881008 0.00124135 2026-02-22T23:45:18.309 INFO:tasks.workunit.client.0.trial122.stdout: 26 15 333911 333896 50.1609 46.457 0.00163395 0.001245 2026-02-22T23:45:19.310 INFO:tasks.workunit.client.0.trial122.stdout: 27 15 346059 346044 50.0604 47.4531 0.00164317 0.0012475 2026-02-22T23:45:20.310 INFO:tasks.workunit.client.0.trial122.stdout: 28 16 359393 359377 50.1325 52.082 0.00112072 0.00124571 2026-02-22T23:45:21.310 INFO:tasks.workunit.client.0.trial122.stdout: 29 16 373022 373006 50.2394 53.2383 0.0010918 0.00124308 2026-02-22T23:45:22.310 INFO:tasks.workunit.client.0.trial122.stdout:Total time run: 30.0009 2026-02-22T23:45:22.310 INFO:tasks.workunit.client.0.trial122.stdout:Total writes made: 386499 2026-02-22T23:45:22.310 INFO:tasks.workunit.client.0.trial122.stdout:Write size: 4096 2026-02-22T23:45:22.310 INFO:tasks.workunit.client.0.trial122.stdout:Object size: 4096 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Bandwidth (MB/sec): 50.3239 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Bandwidth: 3.2563 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Max bandwidth (MB/sec): 55.9453 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Min bandwidth (MB/sec): 46.457 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Average IOPS: 12882 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Stddev IOPS: 833.613 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Max IOPS: 14322 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Min IOPS: 11893 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Average Latency(s): 0.001241 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Stddev Latency(s): 0.000335845 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Max latency(s): 0.00784385 2026-02-22T23:45:22.311 INFO:tasks.workunit.client.0.trial122.stdout:Min latency(s): 0.000483779 2026-02-22T23:45:22.313 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:435: TEST_bluestore_expand: sleep 5 2026-02-22T23:45:27.316 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-22T23:45:27.316 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-22T23:45:27.757 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:437: TEST_bluestore_expand: total_space_before=4294967296 2026-02-22T23:45:27.758 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-22T23:45:27.758 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-22T23:45:27.759 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: grep free 2026-02-22T23:45:27.759 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d: -f 2 2026-02-22T23:45:27.759 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut -d, -f 1 2026-02-22T23:45:27.760 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-22T23:45:27.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:438: TEST_bluestore_expand: free_space_before=2711769088 2026-02-22T23:45:27.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 147135 2026-02-22T23:45:27.836 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-22T23:45:28.837 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 147135 2026-02-22T23:45:28.838 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: sleep 1 2026-02-22T23:45:29.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:441: TEST_bluestore_expand: kill 147135 2026-02-22T23:45:29.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh: line 441: kill: (147135) - No such process 2026-02-22T23:45:29.839 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:442: TEST_bluestore_expand: ceph osd down 0 2026-02-22T23:45:30.036 INFO:tasks.workunit.client.0.trial122.stderr:osd.0 is already down. 2026-02-22T23:45:30.047 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:445: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 allocmap 2026-02-22T23:45:30.064 INFO:tasks.workunit.client.0.trial122.stdout:allocmap bluestore.allocmap 2026-02-22T23:46:13.913 INFO:tasks.workunit.client.0.trial122.stdout:allocmap success 2026-02-22T23:46:13.960 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:448: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:47:00.341 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:47:00.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:450: TEST_bluestore_expand: requested_space=4294967296 2026-02-22T23:47:00.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:451: TEST_bluestore_expand: truncate td/osd-bluefs-volume-ops/0/block -s 4294967296 2026-02-22T23:47:00.390 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:452: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-expand 2026-02-22T23:47:00.410 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:47:41.056 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0x2ca00000(714 MiB) 2026-02-22T23:47:41.057 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0x100000000(4 GiB) : using 0x5e5dd000(1.5 GiB) 2026-02-22T23:47:41.057 INFO:tasks.workunit.client.0.trial122.stdout:Expanding DB/WAL... 2026-02-22T23:47:41.057 INFO:tasks.workunit.client.0.trial122.stdout:1 : nothing to do, skipped 2026-02-22T23:47:41.057 INFO:tasks.workunit.client.0.trial122.stdout:2 : nothing to do, skipped 2026-02-22T23:47:41.774 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:455: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 fsck 2026-02-22T23:48:26.906 INFO:tasks.workunit.client.0.trial122.stdout:fsck success 2026-02-22T23:48:26.950 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:458: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 qfsck 2026-02-22T23:48:26.971 INFO:tasks.workunit.client.0.trial122.stdout:qfsck bluestore.quick-fsck 2026-02-22T23:49:11.838 INFO:tasks.workunit.client.0.trial122.stdout:qfsck success 2026-02-22T23:49:11.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:460: TEST_bluestore_expand: ceph-bluestore-tool --log-file td/osd-bluefs-volume-ops/bluestore_tool.log --path td/osd-bluefs-volume-ops/0 bluefs-bdev-sizes 2026-02-22T23:49:11.901 INFO:tasks.workunit.client.0.trial122.stdout:inferring bluefs devices from bluestore path 2026-02-22T23:49:53.605 INFO:tasks.workunit.client.0.trial122.stdout:1 : device size 0x40000000(1 GiB) : using 0x2ca00000(714 MiB) 2026-02-22T23:49:53.606 INFO:tasks.workunit.client.0.trial122.stdout:2 : device size 0x100000000(4 GiB) : using 0x5e5dd000(1.5 GiB) 2026-02-22T23:49:54.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:462: TEST_bluestore_expand: activate_osd td/osd-bluefs-volume-ops 0 2026-02-22T23:49:54.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:49:54.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: shift 2026-02-22T23:49:54.282 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: local id=0 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: shift 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: local osd_data=td/osd-bluefs-volume-ops/0 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: local 'ceph_args=--fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=9f06c412-45f0-4208-90c5-2f5dfcdd1a2d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd ' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-data=td/osd-bluefs-volume-ops/0' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-journal=td/osd-bluefs-volume-ops/0/journal' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --chdir=' 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+= 2026-02-22T23:49:54.283 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' --run-dir=td/osd-bluefs-volume-ops' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: get_asok_path 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:116: get_asok_path: local name= 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:117: get_asok_path: '[' -n '' ']' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: get_asok_dir 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:120: get_asok_path: echo '/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:863: activate_osd: ceph_args+=' --log-file=td/osd-bluefs-volume-ops/$name.log' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: ceph_args+=' --pid-file=td/osd-bluefs-volume-ops/$name.pid' 2026-02-22T23:49:54.284 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:866: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:868: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:869: activate_osd: ceph_args+=' ' 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph_args+= 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:871: activate_osd: mkdir -p td/osd-bluefs-volume-ops/0 2026-02-22T23:49:54.285 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: echo start osd.0 2026-02-22T23:49:54.286 INFO:tasks.workunit.client.0.trial122.stdout:start osd.0 2026-02-22T23:49:54.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:874: activate_osd: ceph-osd -i 0 --fsid=3501e867-3605-4cb5-a931-33f101f8dade --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=2147483648 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_size=536870912 --bluestore_block_wal_create=true --bluestore_fsck_on_mount=true --bluestore_allocation_from_file=1 --fsid=cbf962cf-8537-48c0-a8a6-b108508f05e4 --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --fsid=9f06c412-45f0-4208-90c5-2f5dfcdd1a2d --auth-supported=none --mon-host=127.0.0.1:7146 --bluestore_block_size=4294967296 --bluestore_block_db_create=true --bluestore_block_db_size=1073741824 --bluestore_block_wal_create=false --bluestore_fsck_on_mount=true --osd_pool_default_size=1 --osd_pool_default_min_size=1 --bluestore_debug_enforce_settings=ssd --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/osd-bluefs-volume-ops/0 --osd-journal=td/osd-bluefs-volume-ops/0/journal --chdir= --run-dir=td/osd-bluefs-volume-ops '--admin-socket=/tmp/ceph-asok.92291/$cluster-$name.asok' --debug-osd=20 '--log-file=td/osd-bluefs-volume-ops/$name.log' '--pid-file=td/osd-bluefs-volume-ops/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-22T23:49:54.287 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: cat td/osd-bluefs-volume-ops/0/whoami 2026-02-22T23:49:54.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:876: activate_osd: '[' 0 = 0 ']' 2026-02-22T23:49:54.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: ceph osd dump --format=json 2026-02-22T23:49:54.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: jq '.flags_set[]' 2026-02-22T23:49:54.287 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:879: activate_osd: grep -q '"noup"' 2026-02-22T23:49:54.306 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:49:54.304+0000 7f4c84c218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:49:54.308 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:49:54.304+0000 7f4c84c218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:49:54.309 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:49:54.304+0000 7f4c84c218c0 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:882: activate_osd: wait_for_osd up 0 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: local state=up 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: local id=0 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:981: wait_for_osd: status=1 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i=0 )) 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:49:54.482 INFO:tasks.workunit.client.0.trial122.stdout:0 2026-02-22T23:49:54.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 0 2026-02-22T23:49:54.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:49:54.483 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:49:54.688 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:49:55.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:49:55.690 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:49:55.690 INFO:tasks.workunit.client.0.trial122.stdout:1 2026-02-22T23:49:55.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 1 2026-02-22T23:49:55.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:49:55.691 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:49:55.783 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:49:55.780+0000 7f4c84c218c0 -1 Falling back to public interface 2026-02-22T23:49:55.901 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:49:56.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:49:56.902 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:49:56.902 INFO:tasks.workunit.client.0.trial122.stdout:2 2026-02-22T23:49:56.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 2 2026-02-22T23:49:56.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:49:56.903 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:49:57.117 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:49:58.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:49:58.118 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:49:58.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 3 2026-02-22T23:49:58.119 INFO:tasks.workunit.client.0.trial122.stdout:3 2026-02-22T23:49:58.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:49:58.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:49:58.317 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:49:59.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:49:59.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:49:59.319 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 4 2026-02-22T23:49:59.319 INFO:tasks.workunit.client.0.trial122.stdout:4 2026-02-22T23:49:59.320 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:49:59.320 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:49:59.517 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:00.518 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:00.518 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:00.519 INFO:tasks.workunit.client.0.trial122.stdout:5 2026-02-22T23:50:00.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 5 2026-02-22T23:50:00.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:00.519 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:00.717 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:01.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:01.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:01.719 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 6 2026-02-22T23:50:01.719 INFO:tasks.workunit.client.0.trial122.stdout:6 2026-02-22T23:50:01.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:01.720 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:01.920 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:02.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:02.922 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:02.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 7 2026-02-22T23:50:02.923 INFO:tasks.workunit.client.0.trial122.stdout:7 2026-02-22T23:50:02.923 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:02.924 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:03.119 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:04.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:04.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:04.121 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 8 2026-02-22T23:50:04.122 INFO:tasks.workunit.client.0.trial122.stdout:8 2026-02-22T23:50:04.122 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:04.122 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:04.325 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:05.326 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:05.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:05.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 9 2026-02-22T23:50:05.327 INFO:tasks.workunit.client.0.trial122.stdout:9 2026-02-22T23:50:05.327 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:05.328 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:05.531 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:06.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:06.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:06.533 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 10 2026-02-22T23:50:06.533 INFO:tasks.workunit.client.0.trial122.stdout:10 2026-02-22T23:50:06.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:06.534 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:06.737 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:07.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:07.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:07.739 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 11 2026-02-22T23:50:07.739 INFO:tasks.workunit.client.0.trial122.stdout:11 2026-02-22T23:50:07.740 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:07.740 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:07.945 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:08.947 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:08.947 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:08.947 INFO:tasks.workunit.client.0.trial122.stdout:12 2026-02-22T23:50:08.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 12 2026-02-22T23:50:08.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:08.948 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:09.156 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:985: wait_for_osd: sleep 1 2026-02-22T23:50:09.624 INFO:tasks.workunit.client.0.trial122.stderr:2026-02-22T23:50:09.620+0000 7f4c84c218c0 -1 osd.0 10 log_to_monitors true 2026-02-22T23:50:10.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i++ )) 2026-02-22T23:50:10.157 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: (( i < 300 )) 2026-02-22T23:50:10.158 INFO:tasks.workunit.client.0.trial122.stdout:13 2026-02-22T23:50:10.158 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:983: wait_for_osd: echo 13 2026-02-22T23:50:10.158 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: ceph osd dump 2026-02-22T23:50:10.159 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:984: wait_for_osd: grep 'osd.0 up' 2026-02-22T23:50:10.362 INFO:tasks.workunit.client.0.trial122.stdout:osd.0 up in weight 1 up_from 12 up_thru 12 down_at 10 last_clean_interval [5,10) [v2:127.0.0.1:6802/1878454372,v1:127.0.0.1:6803/1878454372] [v2:127.0.0.1:6804/1878454372,v1:127.0.0.1:6805/1878454372] exists,up 310a5282-36f1-4995-8a1f-dc926e116260 2026-02-22T23:50:10.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:987: wait_for_osd: status=0 2026-02-22T23:50:10.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:988: wait_for_osd: break 2026-02-22T23:50:10.363 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:991: wait_for_osd: return 0 2026-02-22T23:50:10.363 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: cat td/osd-bluefs-volume-ops/osd.0.pid 2026-02-22T23:50:10.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:463: TEST_bluestore_expand: osd_pid0=151475 2026-02-22T23:50:10.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:465: TEST_bluestore_expand: wait_for_clean 2026-02-22T23:50:10.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1656: wait_for_clean: local cmd= 2026-02-22T23:50:10.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1657: wait_for_clean: local num_active_clean=-1 2026-02-22T23:50:10.365 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: local cur_active_clean 2026-02-22T23:50:10.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: get_timeout_delays 90 .1 2026-02-22T23:50:10.366 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: shopt -q -o xtrace 2026-02-22T23:50:10.366 INFO:tasks.workunit.client.0.trial122.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: echo true 2026-02-22T23:50:10.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1602: get_timeout_delays: local trace=true 2026-02-22T23:50:10.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: true 2026-02-22T23:50:10.366 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1603: get_timeout_delays: shopt -u -o xtrace 2026-02-22T23:50:10.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-22T23:50:10.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1659: wait_for_clean: local -a delays 2026-02-22T23:50:10.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1660: wait_for_clean: local -i loop=0 2026-02-22T23:50:10.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: flush_pg_stats 2026-02-22T23:50:10.465 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2260: flush_pg_stats: local timeout=300 2026-02-22T23:50:10.465 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ceph osd ls 2026-02-22T23:50:10.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2262: flush_pg_stats: ids=0 2026-02-22T23:50:10.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs= 2026-02-22T23:50:10.669 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2264: flush_pg_stats: for osd in $ids 2026-02-22T23:50:10.670 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-22T23:50:10.748 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2265: flush_pg_stats: seq=51539607555 2026-02-22T23:50:10.748 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: test -z 51539607555 2026-02-22T23:50:10.748 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: seqs=' 0-51539607555' 2026-02-22T23:50:10.749 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2273: flush_pg_stats: for s in $seqs 2026-02-22T23:50:10.749 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: echo 0-51539607555 2026-02-22T23:50:10.749 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: cut -d - -f 1 2026-02-22T23:50:10.750 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2274: flush_pg_stats: osd=0 2026-02-22T23:50:10.751 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: echo 0-51539607555 2026-02-22T23:50:10.751 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: cut -d - -f 2 2026-02-22T23:50:10.752 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2275: flush_pg_stats: seq=51539607555 2026-02-22T23:50:10.753 INFO:tasks.workunit.client.0.trial122.stdout:waiting osd.0 seq 51539607555 2026-02-22T23:50:10.753 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2276: flush_pg_stats: echo 'waiting osd.0 seq 51539607555' 2026-02-22T23:50:10.753 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:50:10.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 21474836489 -lt 51539607555 2026-02-22T23:50:10.965 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2278: flush_pg_stats: sleep 1 2026-02-22T23:50:11.966 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2279: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-22T23:50:11.967 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-22T23:50:12.165 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2277: flush_pg_stats: test 51539607555 -lt 51539607555 2026-02-22T23:50:12.165 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-22T23:50:12.166 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:50:12.166 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:50:12.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 == 0 2026-02-22T23:50:12.427 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1667: wait_for_clean: true 2026-02-22T23:50:12.428 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: get_num_active_clean 2026-02-22T23:50:12.428 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1364: get_num_active_clean: local expression 2026-02-22T23:50:12.428 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1365: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-22T23:50:12.428 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1366: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-22T23:50:12.429 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1367: get_num_active_clean: ceph --format json pg dump pgs 2026-02-22T23:50:12.429 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1368: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-22T23:50:12.621 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1671: wait_for_clean: cur_active_clean=16 2026-02-22T23:50:12.622 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: get_num_pgs 2026-02-22T23:50:12.622 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: ceph --format json status 2026-02-22T23:50:12.623 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1425: get_num_pgs: jq .pgmap.num_pgs 2026-02-22T23:50:12.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: test 16 = 16 2026-02-22T23:50:12.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1672: wait_for_clean: break 2026-02-22T23:50:12.881 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1687: wait_for_clean: return 0 2026-02-22T23:50:12.882 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: ceph tell osd.0 perf dump bluefs 2026-02-22T23:50:12.882 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: jq .bluefs.slow_total_bytes 2026-02-22T23:50:12.960 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:467: TEST_bluestore_expand: total_space_after=4294967296 2026-02-22T23:50:12.962 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: ceph tell osd.0 bluestore bluefs device info 2026-02-22T23:50:12.962 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep BDEV_SLOW -A 2 2026-02-22T23:50:12.962 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: grep free 2026-02-22T23:50:12.962 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d: -f 2 2026-02-22T23:50:12.962 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut -d, -f 1 2026-02-22T23:50:12.962 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: cut '-d ' -f 2 2026-02-22T23:50:13.038 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:468: TEST_bluestore_expand: free_space_after=2702028800 2026-02-22T23:50:13.038 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:470: TEST_bluestore_expand: '[' 4294967296 '!=' 4294967296 ']' 2026-02-22T23:50:13.038 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:476: TEST_bluestore_expand: total_space_added=0 2026-02-22T23:50:13.038 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:477: TEST_bluestore_expand: free_space_added=-9740288 2026-02-22T23:50:13.038 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:479: TEST_bluestore_expand: let 'new_used_space=(0 - -9740288)' 2026-02-22T23:50:13.039 INFO:tasks.workunit.client.0.trial122.stdout:9740288 2026-02-22T23:50:13.039 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:480: TEST_bluestore_expand: echo 9740288 2026-02-22T23:50:13.039 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:482: TEST_bluestore_expand: '[' 9740288 -gt 131072 ']' 2026-02-22T23:50:13.039 INFO:tasks.workunit.client.0.trial122.stdout:total_space_added = 0 2026-02-22T23:50:13.039 INFO:tasks.workunit.client.0.trial122.stdout:free_space_added = -9740288 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:483: TEST_bluestore_expand: echo 'total_space_added = 0' 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:484: TEST_bluestore_expand: echo 'free_space_added = -9740288' 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:485: TEST_bluestore_expand: return 1 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh:14: run: return 1 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2379: main: code=1 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2381: main: teardown td/osd-bluefs-volume-ops 1 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:164: teardown: local dir=td/osd-bluefs-volume-ops 2026-02-22T23:50:13.041 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: local dumplogs=1 2026-02-22T23:50:13.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:166: teardown: kill_daemons td/osd-bluefs-volume-ops KILL 2026-02-22T23:50:13.042 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: shopt -q -o xtrace 2026-02-22T23:50:13.042 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: echo true 2026-02-22T23:50:13.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:345: kill_daemons: local trace=true 2026-02-22T23:50:13.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: true 2026-02-22T23:50:13.042 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:346: kill_daemons: shopt -u -o xtrace 2026-02-22T23:50:13.347 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:362: kill_daemons: return 0 2026-02-22T23:50:13.348 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: uname 2026-02-22T23:50:13.349 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:167: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-22T23:50:13.349 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: stat -f -c %T . 2026-02-22T23:50:13.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:168: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-22T23:50:13.351 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: local cores=no 2026-02-22T23:50:13.352 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: sysctl -n kernel.core_pattern 2026-02-22T23:50:13.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:172: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:50:13.353 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:174: teardown: '[' / = '|' ']' 2026-02-22T23:50:13.354 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: grep -q '^core\|core$' 2026-02-22T23:50:13.354 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T23:50:13.355 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:50:13.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: '[' no = yes -o 1 = 1 ']' 2026-02-22T23:50:13.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: '[' -n '' ']' 2026-02-22T23:50:13.356 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:194: teardown: mkdir -p /home/ubuntu/cephtest/archive/log 2026-02-22T23:50:13.357 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:195: teardown: mv td/osd-bluefs-volume-ops/bluestore_tool.log td/osd-bluefs-volume-ops/mgr.x.log td/osd-bluefs-volume-ops/mon.a.log td/osd-bluefs-volume-ops/osd.0.log /home/ubuntu/cephtest/archive/log 2026-02-22T23:50:13.358 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: rm -fr td/osd-bluefs-volume-ops 2026-02-22T23:50:13.424 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: get_asok_dir 2026-02-22T23:50:13.424 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_dir: '[' -n '' ']' 2026-02-22T23:50:13.424 INFO:tasks.workunit.client.0.trial122.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_dir: echo /tmp/ceph-asok.92291 2026-02-22T23:50:13.425 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:199: teardown: rm -rf /tmp/ceph-asok.92291 2026-02-22T23:50:13.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:200: teardown: '[' no = yes ']' 2026-02-22T23:50:13.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:207: teardown: return 0 2026-02-22T23:50:13.426 INFO:tasks.workunit.client.0.trial122.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2382: main: return 1 2026-02-22T23:50:13.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:50:13.428 INFO:tasks.workunit:Stopping ['osd'] on client.0... 2026-02-22T23:50:13.428 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-22T23:50:13.742 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) ^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/workunit.py", line 144, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/workunit.py", line 292, in _spawn_on_all_clients with parallel() as p: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks/workunit.py", line 433, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/remote.py", line 596, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 461, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial122 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh' 2026-02-22T23:50:13.747 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-22T23:50:13.761 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-22T23:50:13.761 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:50:13.869 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-22T23:50:13.881 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-22T23:50:13.881 DEBUG:teuthology.orchestra.run.trial122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:50:13.991 INFO:teuthology.orchestra.run.trial122.stdout: remote refid st t when poll reach delay offset jitter 2026-02-22T23:50:13.991 INFO:teuthology.orchestra.run.trial122.stdout:============================================================================== 2026-02-22T23:50:13.991 INFO:teuthology.orchestra.run.trial122.stdout:-73.185.182.209 204.117.185.216 2 u 63 64 377 44.774 -4.482 2.896 2026-02-22T23:50:13.991 INFO:teuthology.orchestra.run.trial122.stdout:+li1210-167.memb 132.163.96.2 2 u 59 64 377 68.446 -3.519 0.793 2026-02-22T23:50:13.991 INFO:teuthology.orchestra.run.trial122.stdout:+y.ns.gin.ntt.ne 129.250.35.222 2 u 50 64 377 3.115 -3.407 0.595 2026-02-22T23:50:13.991 INFO:teuthology.orchestra.run.trial122.stdout:*ntp.swyn.net 207.66.79.103 2 u 56 64 377 3.511 -2.823 0.631 2026-02-22T23:50:13.992 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-22T23:50:14.003 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-22T23:50:14.003 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-22T23:50:14.020 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-22T23:50:14.037 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-22T23:50:14.051 INFO:teuthology.task.internal:Duration was 1916.327987 seconds 2026-02-22T23:50:14.052 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-22T23:50:14.067 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-22T23:50:14.067 DEBUG:teuthology.orchestra.run.trial122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:50:14.093 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-22T23:50:14.094 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial122.front.sepia.ceph.com 2026-02-22T23:50:14.094 DEBUG:teuthology.orchestra.run.trial122:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:50:14.140 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-22T23:50:14.140 DEBUG:teuthology.orchestra.run.trial122:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:50:14.400 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-22T23:50:14.400 DEBUG:teuthology.orchestra.run.trial122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:50:14.409 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:50:14.410 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:50:14.411 INFO:teuthology.orchestra.run.trial122.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:50:14.411 INFO:teuthology.orchestra.run.trial122.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:50:14.411 INFO:teuthology.orchestra.run.trial122.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:50:14.478 INFO:teuthology.orchestra.run.trial122.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:50:14.480 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-22T23:50:14.495 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-22T23:50:14.496 DEBUG:teuthology.orchestra.run.trial122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:50:14.530 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-22T23:50:14.546 DEBUG:teuthology.orchestra.run.trial122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:50:14.581 INFO:teuthology.orchestra.run.trial122.stdout:kernel.core_pattern = core 2026-02-22T23:50:14.591 DEBUG:teuthology.orchestra.run.trial122:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:50:14.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:50:14.637 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-22T23:50:14.653 INFO:teuthology.task.internal:Transferring archived files... 2026-02-22T23:50:14.656 DEBUG:teuthology.misc:Transferring archived files from trial122:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:40-rados-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64382/remote/trial122 2026-02-22T23:50:14.657 DEBUG:teuthology.orchestra.run.trial122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-23T01:30:45.428 INFO:teuthology.task.internal:Removing archive directory... 2026-02-23T01:30:45.435 DEBUG:teuthology.orchestra.run.trial122:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-23T01:30:49.050 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-23T01:30:49.068 INFO:teuthology.task.internal:Not uploading archives. 2026-02-23T01:30:49.069 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-23T01:30:49.199 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-23T01:30:49.199 DEBUG:teuthology.orchestra.run.trial122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-23T01:30:49.205 INFO:teuthology.orchestra.run.trial122.stdout: 1179687 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Feb 23 01:30 /home/ubuntu/cephtest 2026-02-23T01:30:49.205 INFO:teuthology.orchestra.run.trial122.stdout: 1189028 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 22 23:21 /home/ubuntu/cephtest/mnt.0 2026-02-23T01:30:49.206 INFO:teuthology.orchestra.run.trial122.stdout: 1189029 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 22 23:33 /home/ubuntu/cephtest/mnt.0/client.0 2026-02-23T01:30:49.206 INFO:teuthology.orchestra.run.trial122.stdout: 1203774 4 drwxrwxr-x 3 ubuntu ubuntu 4096 Feb 22 23:33 /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-23T01:30:49.206 INFO:teuthology.orchestra.run.trial122.stdout: 1203775 4 drwxrwxr-x 2 ubuntu ubuntu 4096 Feb 22 23:50 /home/ubuntu/cephtest/mnt.0/client.0/tmp/td 2026-02-23T01:30:49.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-23T01:30:49.207 INFO:teuthology.orchestra.run.trial122.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2026-02-23T01:30:49.208 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 485, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_fa17720d0088c3ac28e473468bfc79eeaff5cd38/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on trial122 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-23T01:30:49.211 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-23T01:30:49.227 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-23T01:30:49.243 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: CommandFailedError: Command failed on trial122 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2026-02-23T01:30:49.249 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} duration: 1916.3279871940613 failure_reason: 'Command failed (workunit test osd/osd-bluefs-volume-ops.sh) on trial122 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/osd/osd-bluefs-volume-ops.sh''' flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com sentry_event: null status: fail success: false 2026-02-23T01:30:49.249 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-23T01:30:49.381 INFO:teuthology.run:FAIL